The “stiffness” of robots communicating with humans is being overcome by technological advances. According to Science and Technology Daily, a Columbia University engineering team announced a major breakthrough on the 15th: the development of a hyper-realistic robot that can “learn” lip movements by observing images. This technology frees robots from the constraints of rigid preset programs, enabling them to precisely mimic mouth shapes when speaking or singing—effectively solving the long-standing problem of ‘fake faces’ that make lifelike robots unsettling.
For a long time, the biggest challenge for realistic robots has been the stiffness when speaking. This robot, developed by Columbia University's Creative Machines Lab, has 26 sets of micro motors installed under the face, covered with flexible materials. The research team’s paper, published in Science Robotics, notes that the robot first “looks in the mirror” to watch itself and understand how its motors control facial expressions. Then, it watches hours of YouTube videos to learn how humans’ mouth shapes change as they talk or sing.
This “Vision-to-Lip Action (VLA)” language model enables the robot to directly convert audio into natural lip movements. During testing, the robot could not only handle multiple languages but even sing AI-generated digital album tracks in time with the music.
●Prediction: A Billion Robots Coming—Lifelike Interaction Is Key for Adoption
Lab director Hod Lipson pointed out that realistic facial expressions are crucial for the future of medical, caregiving, and educational robots. “Robots of the future will definitely have faces,” said Lipson. “If the lip movements aren’t coordinated, they’ll always look unnatural and terrifying.”
Lead researcher Yu-Hang Hu added that this technology, combined with conversational AI like ChatGPT, will give robots facial responses with deeper emotional nuance. As economists predict that over 1 billion humanoid robots may be produced globally in the next decade, scientists emphasize that equipping robots with natural, warm social feedback will be key to humans accepting robots into families and healthcare systems.