Periodic Reporting for period 4 - NatDyReL (Utilizing Natural Dynamics for Reliable Legged Locomotion)
Reporting period: 2023-10-01 to 2024-09-30
The NatDyReL (Utilizing Natural Dynamics for Reliable Legged Locomotion) project aims at a fundamental paradigm shift in the design and control of humanoid robots. In contrast to the now mature technology of torque-controlled drives, the robot developed in NatDyReL will be based on highly compliant actuators. This technology has the strong potential to enable physical robustness against external impacts and allows for periodic energy storage and release during highly dynamic motions. The robot will be able to adapt its dynamic behaviour at runtime to the current ground conditions and to the desired walking speed. In addition, part of the kinetic energy can be temporarily stored in the elastic drives at each step, thus enabling robust and energy-efficient execution of dynamic walking movements. In order to successfully implement these concepts in practice, it is necessary to take the actuator dynamics fully into account in the planning of the overall body movement as well as in the real-time control. Considerable effort thus will be spent on the fusion of whole-body locomotion algorithms with novel concepts for the control of elastic actuators. The project requires close interdisciplinary cooperation between experts from different disciplines, especially from robotics, control engineering and mechatronics.
* In work package 1 we developed a generic control framework for elastic actuators. This framework was shown to be applicable to a wide range of elastic actuators, including position controlled elastic actuators. Moreover, we developed a novel analysis of impactful contact transitions, which lead to the concept of the non-slippage impact direction (NSID). We believe that this concept has applications beyond bipedal locomotion and can be applied also to manipulation tasks such as hammering or sheet metal forming. The detailed treatment of impact-aware motion planning originally was not foreseen in the work plan, but evolved gradually out of the scientific analysis of the locomotion problem.
* In work package 2 we developed algorithms for agile locomotion including motions with flight phases such as hopping and running. We compared both model-based and biologically-inspired algorithms for running and evaluated them on rigid as well as elastic robot models. These detailed simulations were utilized as a basis for the proposed robot design in work package 4. In addition to the development of running algorithms, we extended our DCM-based locomotion framework to include reactive step adaptation in order to react to external disturbances at the body and the feet (stumble reaction). Moreover, we developed an algorithm for the generation and control of angular momentum during balancing and locomotion. Finally, we investigated the combination of model-based locomotion with reinforcement learning in order to further improve the locomotion behavior. One of the challenges of this turned out to be the incompatibility of modern RL algorithms relying on massive parallelization via implementation on GPUs with the computations required for the model based whole body controller. Our solution to this problem was to reduce the number of learned parameters such that an economic reinforcement learning on the CPU was sufficient. This approach allowed to increase the step adaptation to advanced motions without requiring convexity of the step location.
* Work package 3 was devoted to whole-body control and multi-contact interaction. The whole-body control represents the interface between the high-level locomotion strategy (WP2) and the underlying elastic actuator dynamics (WP1). We developed three different whole-body control approaches, including an optimization-based (classic) inverse dynamics WBC and a passivity-based WBC. These algorithms were extended for the treatment of parallel kinematic chains within the robot structure. Such parallel mechanisms are becoming more and more popular in the design of many new humanoid robots, including commercial designs. The performance of WBC for robots driven by actuation with closed kinematic linkages was exemplified via a model of the Kangaroo robot developed by PAL robotics.
* The evaluation of the mentioned concepts was done based on extensive simulations for walking, running, and jumping motions. Based on these results, in work package 4 a detailed design of an elastic humanoid robot was developed. The mechanical design was based on a detailed analysis of the thermal behavior of the actuator units. The analysis lead to a proposed design with an elastic mechanical coupling between the knee and the ankle. For the analyzed running motions it tuned out that a constant elasticity would give a good compromise in terms of speed and torque characteristics of the actuators. The elasticity becomes in particular effective for fast running motions with toe contact, while walking could be implemented on a flat foot contact with minimal disturbance by the elasticity. We believe that this design is a good compromise between controllability and efficiency. In addition to the analysis of the actuator requirements, we also performed a detailed study of the kinematics, based on a comparison with human biomechanics. As a result, the proposed design includes a tilted knee axis having particular benefits for fast running motions with small lateral foot distance.
Exploitation and dissemination of the project results:
The increasing interest of industry in humanoid robots presents a strong potential to further exploit the project results. While dynamic running is not necessarily an important use-case for many practical applications, the developed locomotion concepts extend the general motion skills and robustness of humanoid robots and thus can be interesting for future commercialization. During the project, the results were disseminated in scientific publications as well as keynotes in scientific events.
* In work package 2 we developed algorithms for agile locomotion including motions with flight phases such as hopping and running. We compared both model-based and biologically-inspired algorithms for running and evaluated them on rigid as well as elastic robot models. These detailed simulations were utilized as a basis for the proposed robot design in work package 4. In addition to the development of running algorithms, we extended our DCM-based locomotion framework to include reactive step adaptation in order to react to external disturbances at the body and the feet (stumble reaction). Moreover, we developed an algorithm for the generation and control of angular momentum during balancing and locomotion. Finally, we investigated the combination of model-based locomotion with reinforcement learning in order to further improve the locomotion behavior. One of the challenges of this turned out to be the incompatibility of modern RL algorithms relying on massive parallelization via implementation on GPUs with the computations required for the model based whole body controller. Our solution to this problem was to reduce the number of learned parameters such that an economic reinforcement learning on the CPU was sufficient. This approach allowed to increase the step adaptation to advanced motions without requiring convexity of the step location.
* Work package 3 was devoted to whole-body control and multi-contact interaction. The whole-body control represents the interface between the high-level locomotion strategy (WP2) and the underlying elastic actuator dynamics (WP1). We developed three different whole-body control approaches, including an optimization-based (classic) inverse dynamics WBC and a passivity-based WBC. These algorithms were extended for the treatment of parallel kinematic chains within the robot structure. Such parallel mechanisms are becoming more and more popular in the design of many new humanoid robots, including commercial designs. The performance of WBC for robots driven by actuation with closed kinematic linkages was exemplified via a model of the Kangaroo robot developed by PAL robotics.
* The evaluation of the mentioned concepts was done based on extensive simulations for walking, running, and jumping motions. Based on these results, in work package 4 a detailed design of an elastic humanoid robot was developed. The mechanical design was based on a detailed analysis of the thermal behavior of the actuator units. The analysis lead to a proposed design with an elastic mechanical coupling between the knee and the ankle. For the analyzed running motions it tuned out that a constant elasticity would give a good compromise in terms of speed and torque characteristics of the actuators. The elasticity becomes in particular effective for fast running motions with toe contact, while walking could be implemented on a flat foot contact with minimal disturbance by the elasticity. We believe that this design is a good compromise between controllability and efficiency. In addition to the analysis of the actuator requirements, we also performed a detailed study of the kinematics, based on a comparison with human biomechanics. As a result, the proposed design includes a tilted knee axis having particular benefits for fast running motions with small lateral foot distance.
Exploitation and dissemination of the project results:
The increasing interest of industry in humanoid robots presents a strong potential to further exploit the project results. While dynamic running is not necessarily an important use-case for many practical applications, the developed locomotion concepts extend the general motion skills and robustness of humanoid robots and thus can be interesting for future commercialization. During the project, the results were disseminated in scientific publications as well as keynotes in scientific events.
The project considered a new type of humanoid robot, driven by elastic actuators. The use of elasticity in the actuation is motivated by the possibility to utilize the energy storage in the elastic element for the motion generation and by the capability to protect the actuators against impacts. For locomotion, we developed algorithms for highly dynamic running and we proposed a system design with an elastic coupling between the knee and the ankle. The running algorithms consider both the motion and the interaction forces. By a bio-inspired design, smooth contact force profiles are obtained that are compatible with the elastic actuation. In contrast to the previous state-of-the art, these algorithms allow a precise footstep placement for both walking and running motions. Combinations with machine learning (RL) allow to further optimize the motion for reactive footstep adaptation beyond the algorithmic limitations of a model-based design. These developments can form the basis for future generations of commercial humanoid robot with advanced motion skills and robustness.
For handling impact phenomena, we developed a new analysis of fast contact transitions, leading to the concept of the NSID (non-slippage impact direction). The use of the NSID for manipulation tasks is beyond the scope of the present project and shall be investigated in future actions.
For handling impact phenomena, we developed a new analysis of fast contact transitions, leading to the concept of the NSID (non-slippage impact direction). The use of the NSID for manipulation tasks is beyond the scope of the present project and shall be investigated in future actions.