Robot, object, action!

European researchers at the PACO-PLUS project have developed robotic demonstrators to illustrate a new approach to robotic cognition. The team uses a concept called ‘object-action complexes’ (OACs), pronounced oaks.

These complexes represent a combination of perception of and action upon any given object. The OACs are recorded by the robot every time it performs an action. These recorded actions can be saved and exchanged with other robots, and over time the team hope it will lead to entire libraries of OACs that can be exchanged between researchers.

But that is just a major bonus of this particular system. The primary thrust was to show that robots could teach themselves simple actions, leading to more complex actions and, hopefully, leading to abstract thought in the future.

The PACO-PLUS team performed a large number of demonstrations with humanoid robots and the results so far are very promising.

Autonomous exploration

With autonomous exploration, a robot can patiently explore thousands of potential ways to interact with an object it has no prior knowledge of – learning over time that grasping it in one way is better than another, that it balances better in one direction than another, etc.

“The first thing a robot will do when it encounters an object for the first time is to lift it before its eyes and then rotate the object so the robot gets a look at it from all angles. After that, [the robot] will be able to identify the object from any angle,” explains Tamim Asfour, leader of the Humanoids Research Group at the Institute for Anthropomatics at the Karlsruhe Institute of Technology (KIT) in Germany and co-coordinator of the PACO-PLUS project.

One can see how complex actions could arise over time, as a robot progressively teaches itself how to interact with a cup, jug, handles and doors. Similarly, as the robot’s experience with different objects expands, its options expand, too, and it could learn to use a key with a hole in a door to see if that action will lead to any result.

This is the core of the OAC concept, that interaction with the environment leads to ever-more sophisticated strategies over time, and ultimately gives rise to ‘intelligence’.

Of course, the autonomous exploration is extremely time consuming, unsurprising when one recalls that it took hundreds of millions of years to evolve intelligent life. So another method for teaching robots is learning from human observation and human coaching, and PACO-PLUS developed a large range of successful demonstrators with this method as well.

Human coaching

With coaching, robots can watch humans perform an action, for example wiping a table, and then they can imitate it. It is astonishing how easy it is for robots to mimic complex actions like this.

Similarly, a coach can guide the robots through a complex task, for example putting a cup in a dishwasher. At each stage the coach tells the robot ‘open the dishwasher’, ‘take this object/cup’, ‘put it in the dishwasher’. Over time, the robot learns these sorts of actions.

This is a phenomenally sophisticated demonstration of robotic learning, and it was achieved, using a new approach, in the space of just four years. As such it stands as a remarkable testament to the work ethic of the PACO-PLUS team and the power of its approach.

“Of course, we did a lot of other work too,” notes Asfour. “For example, we looked at some situations with more traditional industrial robots, to see how our work could be applied there.”

The work has been very well received in the wider robotic community, and some partners are looking at how they can integrate some of the results into other projects. In the meantime, PACO-PLUS is currently pursuing further research funding to perfect the work achieved.

But already PACO-PLUS has dramatically advanced the field of robotic cognition and helped to set the research and development agenda for the next decade.

The PACO-PLUS project received funding from the ICT strand of the Sixth Framework Programme for research.

This is the second of a two-part PACO-PLUS feature appearing on ICT Results.

Media note: This feature can be republished without charge provided ICT Results is acknowledged as the source at the top or the bottom of the story. You must request permission before you use any of the photographs on the site. If you do republish, we would be grateful if you could link back to the ICT Results site (http://cordis.europa.eu/ictresults). Let us know if you republish so as to help us provide you with a better service. If you want further contact information on any of the projects cited in this story please contact us.