Periodic Reporting for period 2 - MEMMO (Memory of Motion)
Okres sprawozdawczy: 2019-01-01 do 2020-09-30
What if we could generate complex movements for a robot with any combination of arms and legs interacting with a dynamic environment in real-time? MEMMO has the ambition to create such a motion-generation technology that will revolutionize the motion capabilities of robots and unlock range of industrial and service applications. Based on optimal-control theory, we develop a unified yet tractable approach to motion generation for complex robots. The approach relies on 3 innovative components.
1) a massive amount of pre-computed optimal motions are generated offline and compressed into a "memory of motion".
2) these trajectories are recovered during execution and adapted to new situations with real-time model predictive control, allowing generalization to dynamically changing environments.
3) available sensor modalities (vision, inertial, haptic) are exploited for feedback control which goes beyond the basic robot state with a focus on robust and adaptive behavior.
To demonstrate this approach, MEMMO is organized around 3 relevant industrial applications, where MEMMO technologies have a huge innovation potential. For each application, we will demonstrate the proposed technology in industrial or medical environments, following specifications designed by the end-users partners of the project.
1) A high-performance humanoid robot will perform advanced locomotion and industrial tooling tasks in a 1:1 scale demonstrator of an aircraft assembly.
2) An advanced exoskeleton paired with a paraplegic patient will demonstrate dynamic walking on flat floor, slopes and stairs, in a rehabilitation center under medical surveillance.
3) A challenging inspection task in a real construction site will be performed with a quadruped robot. While challenging, these demonstrators are feasible, as assessed by preliminary results obtained by MEMMO partners.
Memmo is a collaborative project that seeks a breakthrough in the use of complex robots in industrial and medical scenarios. Based on the definition of a new concept, the "memory of motion", it will lead to a new technology for robot control, that has potential to impact several market domains. The variety of impacts is emphasized by demonstrating the technology in 3 relevant environments, defined by our partner stakeholders: PAL ROBOTICS in Barcelona is targeting the market of mobile robots for end-users like AIRBUS; WANDERCRAFT in Paris has designed one of the most advanced exoskeleton for paraplegics, targeting, for the first time, rehabilitation centers like those handle by the Center for Physical Medicine and Rehabilitation of APAJH, based in Pionsat, France; and COSTAIN, a civil-engineering stakeholder, is seeking new solutions for inspection in its constructions sites. The objectives of the project require a collaborative joining expertise in motion planning (brought by LAAS-CNRS, Toulouse, France), robot learning (brought by IDIAP, Switzerland), computer vision (brought by University of Oxford, UK), force control (brought by Max-Planck Institute, Tubingen, Germany and University of Trento, Italy), optimal control (brought by University of Edinbourgh, UK), and capabilities to set up realistic pilot experiments in robotics.
1) a massive amount of pre-computed optimal motions are generated offline and compressed into a "memory of motion".
2) these trajectories are recovered during execution and adapted to new situations with real-time model predictive control, allowing generalization to dynamically changing environments.
3) available sensor modalities (vision, inertial, haptic) are exploited for feedback control which goes beyond the basic robot state with a focus on robust and adaptive behavior.
To demonstrate this approach, MEMMO is organized around 3 relevant industrial applications, where MEMMO technologies have a huge innovation potential. For each application, we will demonstrate the proposed technology in industrial or medical environments, following specifications designed by the end-users partners of the project.
1) A high-performance humanoid robot will perform advanced locomotion and industrial tooling tasks in a 1:1 scale demonstrator of an aircraft assembly.
2) An advanced exoskeleton paired with a paraplegic patient will demonstrate dynamic walking on flat floor, slopes and stairs, in a rehabilitation center under medical surveillance.
3) A challenging inspection task in a real construction site will be performed with a quadruped robot. While challenging, these demonstrators are feasible, as assessed by preliminary results obtained by MEMMO partners.
Memmo is a collaborative project that seeks a breakthrough in the use of complex robots in industrial and medical scenarios. Based on the definition of a new concept, the "memory of motion", it will lead to a new technology for robot control, that has potential to impact several market domains. The variety of impacts is emphasized by demonstrating the technology in 3 relevant environments, defined by our partner stakeholders: PAL ROBOTICS in Barcelona is targeting the market of mobile robots for end-users like AIRBUS; WANDERCRAFT in Paris has designed one of the most advanced exoskeleton for paraplegics, targeting, for the first time, rehabilitation centers like those handle by the Center for Physical Medicine and Rehabilitation of APAJH, based in Pionsat, France; and COSTAIN, a civil-engineering stakeholder, is seeking new solutions for inspection in its constructions sites. The objectives of the project require a collaborative joining expertise in motion planning (brought by LAAS-CNRS, Toulouse, France), robot learning (brought by IDIAP, Switzerland), computer vision (brought by University of Oxford, UK), force control (brought by Max-Planck Institute, Tubingen, Germany and University of Trento, Italy), optimal control (brought by University of Edinbourgh, UK), and capabilities to set up realistic pilot experiments in robotics.
The consortium is now working as a strong research group, organized around several weekly workshops and open software collaboration. We have simplified the experimental process and now share the experimental work around 3 Talos (CNRS, PAL, UEDIN), 4 Anymal robots (UOXF, UEDIN), several Atalante (v1 and v2 at WAN) and several experimental setups from the Open Dynamic Robot Initiative (MPI, CNRS, UNITN and next UEDIN, UOXF and PAL).
The main theoretical tools have been released during the first period, in particular the optimal control software Crocoddyl and first memories of motion but also other softwares for robot estimation, localization and mapping, low-level control, new mechatronic design, etc. We have produced the datasets for several experimental scenarios, benchmarked several learning formulation to encode the memory.
The 2nd period has been devoted to the first experimental proofs on real hardware. We have demonstrated the first whole-body model predictive controller with low-level torque servo on Talos at CNRS and PAL, for manipulation and obstacle avoidance. The controller uses both our new solver, able to compute an optimization step for 10K variables every 10ms, and a memory of motion trained off-line which guides the convergence of the solver. Simultaneously, several experimental achievements have been obtained in preparation for the last phase of the project, validating the experimental setups and scenarios. We have implemented and released templated controllers for our quadruped and biped, balance control for our exoskeleton, demonstrated localization and mapping with laser and RGBD cameras on both the quadruped Anymal and the biped Talos.
We have also demonstrated our robots performing realistic actions in the industrial or medical contexts for our 3 case studies: deburing with a mobile manipulator for aerospace manufacturing; participation to the cybathlon with several challenging new tasks with the exoskeleton; navigation on an industrial site with the quadruped. Our next challenge is to demonstrate the whole-body model predictive control for achieving locomotion, first on quadrupeds.
The main theoretical tools have been released during the first period, in particular the optimal control software Crocoddyl and first memories of motion but also other softwares for robot estimation, localization and mapping, low-level control, new mechatronic design, etc. We have produced the datasets for several experimental scenarios, benchmarked several learning formulation to encode the memory.
The 2nd period has been devoted to the first experimental proofs on real hardware. We have demonstrated the first whole-body model predictive controller with low-level torque servo on Talos at CNRS and PAL, for manipulation and obstacle avoidance. The controller uses both our new solver, able to compute an optimization step for 10K variables every 10ms, and a memory of motion trained off-line which guides the convergence of the solver. Simultaneously, several experimental achievements have been obtained in preparation for the last phase of the project, validating the experimental setups and scenarios. We have implemented and released templated controllers for our quadruped and biped, balance control for our exoskeleton, demonstrated localization and mapping with laser and RGBD cameras on both the quadruped Anymal and the biped Talos.
We have also demonstrated our robots performing realistic actions in the industrial or medical contexts for our 3 case studies: deburing with a mobile manipulator for aerospace manufacturing; participation to the cybathlon with several challenging new tasks with the exoskeleton; navigation on an industrial site with the quadruped. Our next challenge is to demonstrate the whole-body model predictive control for achieving locomotion, first on quadrupeds.
In the beginning of the 3rd period, the consortium is focusing on extending the proposed control for locomotion. For that, we need to experimentally validate the low-level servo and the whole body predictive control. The optimal-control formulation for locomotion has already been chosen and validated in simulation. The process to training process is now mature and new memories can be produced on demand for new experimental contexts. We are also currently exploring how to robustify the solver and best exploit the memory of motion to enable the solver to tackle more complex problems, e.g. walking above obstacles or in stairs.
The main concepts are working and the experimental setups are clear. We now incrementally complexity the experimental tasks in order to reach the three industrial and medical case-studies. The road is paved by early validations with more classical control laws that enabled us to validate the robots and foreseen scenarios. We have defined the task to be achieved in aerospace manufacturing context provided AIRBUS: navigating to the work bench and deburing a set of holes with the humanoid robot while replaning on-the-flight in case of perturbation. We have defined the set of tasks to be achieved in the medical context provided by WANDERCRAFT: they have been benchmark in realistic condition during the last edition of the Cybathlone. We also have defined the set of tasks to be achieved in a context of inspecting civil-engineering structures: thee quadruped will have to navigate in a training industrial facility.
For all these scenarios, we have some partial validation using ad-hoc controllers, and preliminary experimental validation that the proposed whole-body predictive controller with a memory of motion is able to tackle them better. The work of the final period will be to confirm these preliminary experimental results, and demonstrate that our contributions scale to realistic industrial and medical scenarios.
The main concepts are working and the experimental setups are clear. We now incrementally complexity the experimental tasks in order to reach the three industrial and medical case-studies. The road is paved by early validations with more classical control laws that enabled us to validate the robots and foreseen scenarios. We have defined the task to be achieved in aerospace manufacturing context provided AIRBUS: navigating to the work bench and deburing a set of holes with the humanoid robot while replaning on-the-flight in case of perturbation. We have defined the set of tasks to be achieved in the medical context provided by WANDERCRAFT: they have been benchmark in realistic condition during the last edition of the Cybathlone. We also have defined the set of tasks to be achieved in a context of inspecting civil-engineering structures: thee quadruped will have to navigate in a training industrial facility.
For all these scenarios, we have some partial validation using ad-hoc controllers, and preliminary experimental validation that the proposed whole-body predictive controller with a memory of motion is able to tackle them better. The work of the final period will be to confirm these preliminary experimental results, and demonstrate that our contributions scale to realistic industrial and medical scenarios.