Meet the robot that talks likes us
Human-looking robots will become more and more commonplace in our daily lives. It’s understandable then that scientists want to make them look as human as possible. If they look like us, why not speak like us, too? A team of scientists led by Columbia University in the United States created a robot named Emo(opens in new window) that is able to learn how to move its lips for speaking and singing. The findings were published in the journal ‘Science Robotics’(opens in new window).
Learning to lip sync to speech and song
At first, they designed Emo to practice in front of a mirror, experimenting with its 26 facial muscles to help it learn how its own face moves. Then it watched hours of YouTube videos to observe how human mouths move during speech and song. The face, made with silicone skin, is able to form lip shapes that cover 24 consonants and 16 vowels. “The more it interacts with humans, the better it will get,” commented engineering professor Hod Lipson in a news release(opens in new window). “When the lip sync ability is combined with conversational AI such as ChatGPT or Gemini, the effect adds a whole new depth to the connection the robot forms with the human,” explained Yuhang Hu, a researcher at Creative Machines Lab where the work was carried out. “The more the robot watches humans conversing, the better it will get at imitating the nuanced facial gestures we can emotionally connect with.”
Missing piece to robotics?
“Much of humanoid robotics today is focused on leg and hand motion, for activities like walking and grasping,” elaborated Lipson. “But facial affection is equally important for any robotic application involving human interaction.” He added: “There is no future where all these humanoid robots don’t have a face. And when they finally have a face, they will need to move their eyes and lips properly, or they will forever remain uncanny.” Lipson is excited at the prospect of a robot feeling more natural. “Something magical happens when a robot learns to smile or speak just by watching and listening to humans. I’m a jaded roboticist, but I can’t help but smile back at a robot that spontaneously smiles at me.” Human faces are the most powerful medium for communication. This will become even more important for advanced humanoid machines. “Robots with this ability will clearly have a much better ability to connect with humans because such a significant portion of our communication involves facial body language, and that entire channel is still untapped,” stated Hu. However, Lipson is aware of the risks and potential controversies in creating robots that are more natural and emotionally engaging. “This will be a powerful technology. We have to go slowly and carefully, so we can reap the benefits while minimizing the risks.” Sooner or later, humanoid robots will talk – and sing – in ways that seem so natural. Hopefully, we’ll never mistake that we’re communicating with a robot, and not a person.