Skip to main content
European Commission logo print header

Article Category

Content archived on 2023-03-02

Article available in the following languages:


Teachable servants

What's the easiest way of telling a robot what to do? It’s using words and gestures. A robot that discerns and imitates its operator's arm movements with the aid of a 3D camera has proved that gesture recognition is feasible.

The fellow is a truly fearsome sight: as tall as a basketball player, with a bald bluish head and steel extremities that remind you of a skeleton. "Mirrobot" causes heads to turn wherever he goes. "A crowd gathers every time the robot moves", says Ulrich Reiser, who presented the steel assistant to the public at CeBit in March and is now taking it to Automatica. He and his colleagues breathed artificial life into the machine, a development by Festo of Esslingen. The researchers of the Stuttgart-based Fraunhofer Institute for Manufacturing Engineering and Automation IPA have made Mirrobot imitate the actions of a human counterpart. As soon as the person raises an arm, the robot moves its arm too. If the person stretches both arms sideways, Mirrobot does the same thing – as though it were a live mirror image. What looks like a gimmick is actually trailblazing research. It's all about controlling robots. The goal of the Fraunhofer experts is to create an automatic assistant that can vacuum the carpet, fetch the newspaper and cook pasta when told to do so – a servant that can do all the housework, in fact. One of the biggest challenges that this involves is human-machine interaction: How do you tell the robot what to do? It is relatively simple to write a computer program for industrial robots that constantly repeat the same maneuvers. It is quite a different matter for a service robot that has to perform reliably in a constantly changing environment amid the domestic clutter of someone's home. Ideally, you ought to be able to instruct the robot in a simple and intuitive way: by pointing to the table, for instance, and saying "Bring me the newspaper!" Whilst there are already voice recognition programs on the market, machines are not yet capable of understanding gestures. But Mirrobot shows that this problem can be solved too. A novel miniature camera helps the robot to interpret gestures: it measures distances by recording the time traveled by infrared rays, thus delivering a 3D picture of the environment. From this, an image processing program determines the spatial position of the user's hands and head. But this is only the first step. In the second step, the robot has to learn the meaning of certain gestures when accompanied by spoken commands. It will still be a long time before an affordable service robot goes onto the market, but IPA team leader Achim Breckweg is convinced that as time progresses, service robots will be able to take over increasingly complex household tasks.


Austria, Belgium, Czechia, Germany, Denmark, Estonia, Greece, Spain, Finland, France, Hungary, Ireland, Italy, Lithuania, Luxembourg, Latvia, Malta, Netherlands, Poland, Portugal, Sweden, Slovenia, Slovakia, United Kingdom