Even after 50 years of robotics research, most existing robots are far from being as resilient as even the smallest, simplest animals – in short, they can easily stop functioning if they encounter difficult conditions or take an unexpected tumble. For robots to become more embedded in human societies and for them to be useful in emergency situations, they need to overcome their inherent frailness. That’s where the ResiBots (Robots with animal-like resilience) project comes in. “The objective of our project was to develop new algorithms to make it possible for robots to autonomously recover quickly without having to anticipate every possible damage condition,” says Jean-Baptiste Mouret, the Principal Investigator of the project. “This is especially important when they involve mechanical damage that is very challenging to diagnose with onboard sensors – our vision is to develop robots that have the ability to ‘improvise’ when they need to complete a difficult mission.”
Adopting a new way of thinking for more resilient robots
This vision is in direct contrast to the current approach to fault tolerance, which is inherited from safety-critical systems (such as for spacecraft or nuclear power plants). This is inappropriate for low-cost autonomous robots because it relies on diagnosis procedures, requiring expensive proprioceptive sensors and contingency plans – and these cannot cover all the possible situations that an autonomous robot operating on its own can encounter. However, the algorithms developed by Mouret and his team circumvent this older-style thinking as they are specifically designed for data-efficient (aka with very little data) adaptation in robotics. “For instance, our algorithms allow a six-legged robot to find a new walking gait in less than 2 minutes,” Mouret explains. “Overall, the general idea is to leverage a simulation of the ‘intact’ robot to accelerate the adaptation for a robot with unknown damage.”
From diagnosis to reinforcement algorithms
Importantly for the project, this first result uses ‘episodic learning’, which means that each trial starts in exactly the same position. A more recent algorithm allows the robot to learn independently without any reset, whilst taking its environment into account. “For instance, there is no reason for the robot to try a gait that is likely to make it move forward if we know there is an obstacle in the way,” says Mouret. As he argues, machines – and particularly robots – have been essentially ‘blind’ thus far, where easy tasks for animals (such as recognising an object) have proven too difficult for robots’ vision algorithms. But thanks to recent advances in deep learning, machines are now able to understand their surroundings much better, opening the gates to many possible applications, including those that are beyond the factory floor. “However, if we want robots that can truly learn new skills and/or tasks, they need reinforcement algorithms and not only perception algorithms,” warns Mouret. “For this to happen, we need to make them more data-efficient before they can be deployed outside research labs.” He does not envision the deployment of robots that can learn to undertake new tasks by themselves within the next 5 years but is optimistic that it could happen within the next 5 to 15 years.
Looking to the future
ResiBots has truly been a trailblazer in lighting the path to reach this goal and Mouret and his team plan to keep their research going even after the project’s official end in April 2020. “We want to introduce trial-and-error learning to humanoid robotics – humanoid robots are the ‘next frontier’ of robotics because they combine all of the challenges of robotics at the same time,” says Mouret. These challenges include the need to synchronise many joints (typically 24) and they need to react quickly and keep their balance. Whilst no current humanoid robot is currently using trial-and-error learning to adapt to changes in its environment, Mouret and his team want to be at the forefront of changing this. And when asked what he is most proud of from ResiBots? “We routinely have robots that learn to walk right in front of our eyes, in just a few minutes, compared to hours or even days before,” Mouret concludes. “I have to say, these were the kinds of robots I really dreamed about when I was a child!”
ResiBots, robots, algorithms, episodic learning, data-efficient