Periodic Reporting for period 3 - MEMMO (Memory of Motion)
Reporting period: 2020-10-01 to 2022-06-30
What if we could generate complex movements for a robot with any combination of arms and legs interacting with a dynamic environment in real-time? MEMMO has the ambition to create such a motion-generation technology that will revolutionize the motion capabilities of robots and unlock range of industrial and service applications. Based on optimal-control theory, we develop a unified yet tractable approach to motion generation for complex robots. The approach relies on 3 innovative components.
1) a massive amount of pre-computed optimal motions are generated offline and compressed into a "memory of motion".
2) these trajectories are recovered during execution and adapted to new situations with real-time model predictive control, allowing generalization to dynamically changing environments.
3) available sensor modalities (vision, inertial, haptic) are exploited for feedback control which goes beyond the basic robot state with a focus on robust and adaptive behavior.
To demonstrate this approach, MEMMO is organized around 3 relevant industrial applications, where MEMMO technologies have a huge innovation potential. For each application, we will demonstrate the proposed technology in industrial or medical environments, following specifications designed by the end-users partners of the project.
1) A high-performance humanoid robot will perform advanced locomotion and industrial tooling tasks in a 1:1 scale demonstrator of an aircraft assembly.
2) An advanced exoskeleton paired with a paraplegic patient will demonstrate dynamic walking on flat floor, slopes and stairs, in a rehabilitation center under medical surveillance.
3) A challenging inspection task in a real construction site will be performed with a quadruped robot. While challenging, these demonstrators are feasible, as assessed by preliminary results obtained by MEMMO partners.
Memmo is a collaborative project that seeks a breakthrough in the use of complex robots in industrial and medical scenarios. Based on the definition of a new concept, the "memory of motion", it will lead to a new technology for robot control, that has potential to impact several market domains. The variety of impacts is emphasized by demonstrating the technology in 3 relevant environments, defined by our partner stakeholders: PAL ROBOTICS in Barcelona is targeting the market of mobile robots for end-users like AIRBUS; WANDERCRAFT in Paris has designed one of the most advanced exoskeleton for paraplegics, targeting, for the first time, rehabilitation centers like those handle by the Center for Physical Medicine and Rehabilitation of APAJH, based in Pionsat, France; and COSTAIN, a civil-engineering stakeholder, is seeking new solutions for inspection in its constructions sites. The objectives of the project require a collaborative joining expertise in motion planning (brought by LAAS-CNRS, Toulouse, France), robot learning (brought by IDIAP, Switzerland), computer vision (brought by University of Oxford, UK), force control (brought by Max-Planck Institute, Tubingen, Germany and University of Trento, Italy), optimal control (brought by University of Edinbourgh, UK), and capabilities to set up realistic pilot experiments in robotics.
1) a massive amount of pre-computed optimal motions are generated offline and compressed into a "memory of motion".
2) these trajectories are recovered during execution and adapted to new situations with real-time model predictive control, allowing generalization to dynamically changing environments.
3) available sensor modalities (vision, inertial, haptic) are exploited for feedback control which goes beyond the basic robot state with a focus on robust and adaptive behavior.
To demonstrate this approach, MEMMO is organized around 3 relevant industrial applications, where MEMMO technologies have a huge innovation potential. For each application, we will demonstrate the proposed technology in industrial or medical environments, following specifications designed by the end-users partners of the project.
1) A high-performance humanoid robot will perform advanced locomotion and industrial tooling tasks in a 1:1 scale demonstrator of an aircraft assembly.
2) An advanced exoskeleton paired with a paraplegic patient will demonstrate dynamic walking on flat floor, slopes and stairs, in a rehabilitation center under medical surveillance.
3) A challenging inspection task in a real construction site will be performed with a quadruped robot. While challenging, these demonstrators are feasible, as assessed by preliminary results obtained by MEMMO partners.
Memmo is a collaborative project that seeks a breakthrough in the use of complex robots in industrial and medical scenarios. Based on the definition of a new concept, the "memory of motion", it will lead to a new technology for robot control, that has potential to impact several market domains. The variety of impacts is emphasized by demonstrating the technology in 3 relevant environments, defined by our partner stakeholders: PAL ROBOTICS in Barcelona is targeting the market of mobile robots for end-users like AIRBUS; WANDERCRAFT in Paris has designed one of the most advanced exoskeleton for paraplegics, targeting, for the first time, rehabilitation centers like those handle by the Center for Physical Medicine and Rehabilitation of APAJH, based in Pionsat, France; and COSTAIN, a civil-engineering stakeholder, is seeking new solutions for inspection in its constructions sites. The objectives of the project require a collaborative joining expertise in motion planning (brought by LAAS-CNRS, Toulouse, France), robot learning (brought by IDIAP, Switzerland), computer vision (brought by University of Oxford, UK), force control (brought by Max-Planck Institute, Tubingen, Germany and University of Trento, Italy), optimal control (brought by University of Edinbourgh, UK), and capabilities to set up realistic pilot experiments in robotics.
The consortium is now working as a strong research group, organized around several weekly workshops and open software collaboration. We have simplified the experimental process and now share the experimental work around 3 Talos (CNRS, PAL, UEDIN), 4 Anymal robots (UOXF, UEDIN), several Atalante (v1 and v2 at WAN) and several experimental setups from the Open Dynamic Robot Initiative (MPI, CNRS, UNITN and next UEDIN, UOXF and PAL).
The main theoretical tools have been released during the first period, in particular the optimal control software Crocoddyl and first memories of motion but also other softwares for robot estimation, localization and mapping, low-level control, new mechatronic design, etc. We have produced the datasets for several experimental scenarios, benchmarked several learning formulation to encode the memory. The second period has been used for the preliminary experimental validations. The third period has been dedicated to the final definition of the industrial and medical scenarios and the implementation of the proposed method on the final demonstrators.
The main theoretical tools have been released during the first period, in particular the optimal control software Crocoddyl and first memories of motion but also other softwares for robot estimation, localization and mapping, low-level control, new mechatronic design, etc. We have produced the datasets for several experimental scenarios, benchmarked several learning formulation to encode the memory. The second period has been used for the preliminary experimental validations. The third period has been dedicated to the final definition of the industrial and medical scenarios and the implementation of the proposed method on the final demonstrators.
Based on the expertise brought by all the partners, we have successfully implemented the full concept of the project, namely a whole-body model predictive controller with a memory of motion on real robots. The interest of the approach has been demonstrated on 3 industrial and medical scenarios. The work has led to 7 innovations either commercialized or proposed as open-source software or hardware concepts to the community. The project also led to the publication of more than 130 peer-reviewed international publications (rank A, among which more than 35 journals), 7 workshops in international conferences and 10 keynotes. A new company, Toward, was open in Toulouse, France, to exploit the results of the project, along with two joint public/private laboratory (ROB4FAM with Airbus and CNRS, started 2017, renewed 2022 ; and DynamoGrade with Toward and CNRS, started 2022).