Periodic Reporting for period 4 - LeMo (Learning Mobility for Real Legged Robots)
Período documentado: 2024-07-01 hasta 2024-12-31
LEMO investigated the use of machine learning technologies to increase the mobility of legged robotic systems, enabling such systems to achieve unprecedented locomotion and navigation skills in challenging environments. We leveraged simulation tools paired with models from the real world to generate the data necessary for training locomotion policies that can be transferred and deployed on real systems. The work performed in this ERC has pioneered and contributed to the transformation of control of legged robots from classic model-based approaches to reinforcement learning with simulation-based data generation. It provides solutions for closing the reality gap and tightly integrating perception with locomotion and navigation. Our works have continuously demonstrated state-of-the-art locomotion and navigation performance with quadrupedal robots, highlighted through a series of Science Robotics papers and real-world demonstrations.
Our robots and control approaches have been used for example during the DARPA SubT Competition, which our team won in 2021, and they made their way to industrial applications and search and rescue demonstrations.
While the technology is mostly tested and demonstrated on a series of quadrupedal robots and in the context of locomotion, the approach, and developed tools are transferable to other systems and problems. We demonstrated, for example, the application of autonomous earthwork with automated excavators.
Our robots and control approaches have been used for example during the DARPA SubT Competition, which our team won in 2021, and they made their way to industrial applications and search and rescue demonstrations.
While the technology is mostly tested and demonstrated on a series of quadrupedal robots and in the context of locomotion, the approach, and developed tools are transferable to other systems and problems. We demonstrated, for example, the application of autonomous earthwork with automated excavators.
The ERC project pioneered the transition from classical model-based control to reinforcement learning (RL)-based control for legged robots. We developed simulation-to-reality pipelines for training robust locomotion policies entirely in simulation, which were successfully deployed on physical quadrupeds for agile and versatile locomotion in diverse environments. These breakthroughs were demonstrated in high-profile venues, including the DARPA SubT Challenge (which our team won in 2021) and ESA’s planetary exploration trials.
Our research contributed to over 45 publications, including a series for Science Robotics publications, five PhD theses, and multiple open-source tools such as LeggedGym and Orbit, now commercialized and adopted widely by academia and industry (e.g. NVIDIA’s Isaac Lab). Companies such as ANYbotics and others have integrated our tools and approaches into their products. The developed methods extended beyond quadrupeds to wheeled-legged robots, mobile manipulators, and autonomous excavators, highlighting their broad applicability.
Dissemination efforts included keynote talks at top robotics conferences (ICRA, IROS, GTC, CES), open-source releases, and collaborations with companies such as NVIDIA. The work led to the founding of startups like Gravis Robotics and RIVR, directly transferring technology to industrial applications including autonomous construction and last-mile delivery.
Our research contributed to over 45 publications, including a series for Science Robotics publications, five PhD theses, and multiple open-source tools such as LeggedGym and Orbit, now commercialized and adopted widely by academia and industry (e.g. NVIDIA’s Isaac Lab). Companies such as ANYbotics and others have integrated our tools and approaches into their products. The developed methods extended beyond quadrupeds to wheeled-legged robots, mobile manipulators, and autonomous excavators, highlighting their broad applicability.
Dissemination efforts included keynote talks at top robotics conferences (ICRA, IROS, GTC, CES), open-source releases, and collaborations with companies such as NVIDIA. The work led to the founding of startups like Gravis Robotics and RIVR, directly transferring technology to industrial applications including autonomous construction and last-mile delivery.
The project significantly advanced the state of the art by demonstrating that learning-based controllers, trained entirely in simulation, can outperform traditional methods in robustness, versatility, and real-world deployment. It introduced novel training paradigms—such as terrain curricula, hierarchical navigation structures, and perceptive control using multimodal sensors—and addressed the sim-to-real gap at scale.
The developed technologies enabled the first commercial AI-controlled quadrupedal robots and set new benchmarks in field robotics, navigation, and manipulation. Progress has already extended into semantic scene understanding using LLMs/VLMs, enabling legged robots to autonomously navigate complex, unknown environments.
While the core project has concluded, its momentum continues. Ongoing work includes expanding into humanoid robotics, further industrial collaborations, and scaling learned control to more complex, semantically rich tasks in dynamic and unstructured environments.
The developed technologies enabled the first commercial AI-controlled quadrupedal robots and set new benchmarks in field robotics, navigation, and manipulation. Progress has already extended into semantic scene understanding using LLMs/VLMs, enabling legged robots to autonomously navigate complex, unknown environments.
While the core project has concluded, its momentum continues. Ongoing work includes expanding into humanoid robotics, further industrial collaborations, and scaling learned control to more complex, semantically rich tasks in dynamic and unstructured environments.