Periodic Reporting for period 3 - DARKO (Dynamic Agile Production Robots That Learn and Optimise Knowledge and Operations)
Okres sprawozdawczy: 2024-01-01 do 2025-06-30
The DARKO project addresses the critical challenge of enhancing the efficiency and safety of dynamic mobile manipulation, an area of significant importance due to its potential impact on society. It focuses on developing advanced robotic systems capable of performing dynamic tasks—such as picking, placing, and throwing objects—with greater precision and safety. The project's objectives are geared towards creating solutions that facilitate the seamless integration of robots into environments shared with humans, thereby revolutionizing industries like intralogistics and manufacturing. This is particularly important for society as it not only promises to improve productivity and reduce operational costs but also ensures the safety and well-being of human workers in close proximity to robots. The objectives of the DARKO project include:
O1- Efficient and Safe Dynamic Mobile Manipulation: Improve the efficiency and safety of robots in dynamic tasks through the use of elastic manipulators and high-speed perception technologies.
O2- Predictive Safety and Efficiency in Human-Robot Coordination**: Develop models that predict and enhance the safety and efficiency of human-robot interactions, ensuring seamless integration in shared environments.
O3- Efficient Deployment and Safe Localization**: Streamline the deployment of intralogistics robots with minimal reliance on labeled data by employing semi-supervised and self-supervised learning for robust mapping and localization.
O4- Risk-aware Operation for Safety and Efficiency**: Integrate risk assessment into robotic operations to manage the risks associated with robot actions, improving both safety and operational efficiency.
O1- Efficient and Safe Dynamic Mobile Manipulation: Improve the efficiency and safety of robots in dynamic tasks through the use of elastic manipulators and high-speed perception technologies.
O2- Predictive Safety and Efficiency in Human-Robot Coordination**: Develop models that predict and enhance the safety and efficiency of human-robot interactions, ensuring seamless integration in shared environments.
O3- Efficient Deployment and Safe Localization**: Streamline the deployment of intralogistics robots with minimal reliance on labeled data by employing semi-supervised and self-supervised learning for robust mapping and localization.
O4- Risk-aware Operation for Safety and Efficiency**: Integrate risk assessment into robotic operations to manage the risks associated with robot actions, improving both safety and operational efficiency.
The DARKO project has made significant advancements across four main objectives: dynamic mobile manipulation, predictive safety and efficiency in human-robot coordination, efficient deployment and safe localization, and risk-aware operation for safety and efficiency. Here's a summary of the main work done and the key results in each area:
O1: Efficient and Safe Dynamic Mobile Manipulation
- Main Work: Enhancements in dynamic manipulation tasks were achieved through the development of inherently elastic manipulators, advanced high-speed perception capabilities, and algorithms to improve efficiency and safety.
- Key Results: Improved performance of the SoftHand, a new elastic wrist design, development of modular Bi-Stiffness Actuators (BSA), and comprehensive controller packages for the Franka Emika Panda. integration of BSA into an elastic robotic arm. Innovative methods (e.g.Thunder Dynamics library, adaptive torque control, learning-based and dictionnary-based methods) for robust throwing motions and human-like throwing strategies were also introduced. Finally, a 360-degree object detection with fish-eye camera and lidar input was developed for efficient detection of objects to be manipulated.
O2: Predictive Safety and Efficiency in Human-Robot Coordination
- Main Work: Focused on integrating robots into existing warehouse operations safely and efficiently, utilizing long-term human motion prediction methods, and developing a multi-modal motion capture dataset.
- Key Results: Enhanced human-robot interaction through advanced motion prediction algorithms, integration of gaze tracking technology, vocal interface, and the deployment of the Anthropomorphic Robot Mock Driver (ARMoD). Introduction of neuro-symbolic architecture for motion prediction and implementation of a human-aware navigation system. Development of the THOR-MAGNI Act dataset with action-level annotations for training activity aware models, even in occlusion-prone environments.
O3: Efficient Deployment and Safe Localization
- Main Work: Aimed at creating mapping and localization systems that are failure-aware and resilient, focusing on data-efficient perception methods to reduce the need for extensive semantic annotations.
- Key Results: Creation of an annotated 3D intralogistics dataset, deployment of the multi-class 9DoF RGB-D YOLO++ detector, and advancements in grasp pose estimation and object detection technologies. Significant improvements in human perception operations and the development of a novel neural surface representation for efficient and accurate 3D reconstruction. Development of a pipeline for real-time large-scale 3D mapping, efficient and consistent even with uncalibrated video footage.
O4: Risk-aware Operation for Safety and Efficiency
- Main Work: Incorporation of risk assessment into robotic decision-making, focusing on predicting and mitigating potential risks to ensure safety and operational efficiency.
- Key Results: Development of methodologies for anticipating navigation and manipulation risks, continuous monitoring (with causal reasoning algorithms) of risk levels during operations, and the implementation of a Stochastic Dynamic Programming engine (and task scheduler) to optimize decision-making under uncertainty. Development and integration of a Safe Motion Unit aligned with latest safety standards.
These achievements collectively enhance the capabilities of robotic systems in dynamic manipulation, human-robot interaction, autonomous navigation, and risk-aware operation, setting new standards for safety, efficiency, and technological innovation in the field of intralogistics robotics.
At the project end, two integrated mobile robotic platform have been developed and demonstrated to academic and industrial stakeholders, and have been showcased at a large open event (Automatica 2025, Munich). The system was capable of autonomously map the environment, interact efficiently with nearby humans, identify, grasp and throw objects to a dedicated target area. The consortium developed open-source libraries that can be used freely in the future. The project led to an italian patent, and extends the product portfolio of one of the industrial partners.
O1: Efficient and Safe Dynamic Mobile Manipulation
- Main Work: Enhancements in dynamic manipulation tasks were achieved through the development of inherently elastic manipulators, advanced high-speed perception capabilities, and algorithms to improve efficiency and safety.
- Key Results: Improved performance of the SoftHand, a new elastic wrist design, development of modular Bi-Stiffness Actuators (BSA), and comprehensive controller packages for the Franka Emika Panda. integration of BSA into an elastic robotic arm. Innovative methods (e.g.Thunder Dynamics library, adaptive torque control, learning-based and dictionnary-based methods) for robust throwing motions and human-like throwing strategies were also introduced. Finally, a 360-degree object detection with fish-eye camera and lidar input was developed for efficient detection of objects to be manipulated.
O2: Predictive Safety and Efficiency in Human-Robot Coordination
- Main Work: Focused on integrating robots into existing warehouse operations safely and efficiently, utilizing long-term human motion prediction methods, and developing a multi-modal motion capture dataset.
- Key Results: Enhanced human-robot interaction through advanced motion prediction algorithms, integration of gaze tracking technology, vocal interface, and the deployment of the Anthropomorphic Robot Mock Driver (ARMoD). Introduction of neuro-symbolic architecture for motion prediction and implementation of a human-aware navigation system. Development of the THOR-MAGNI Act dataset with action-level annotations for training activity aware models, even in occlusion-prone environments.
O3: Efficient Deployment and Safe Localization
- Main Work: Aimed at creating mapping and localization systems that are failure-aware and resilient, focusing on data-efficient perception methods to reduce the need for extensive semantic annotations.
- Key Results: Creation of an annotated 3D intralogistics dataset, deployment of the multi-class 9DoF RGB-D YOLO++ detector, and advancements in grasp pose estimation and object detection technologies. Significant improvements in human perception operations and the development of a novel neural surface representation for efficient and accurate 3D reconstruction. Development of a pipeline for real-time large-scale 3D mapping, efficient and consistent even with uncalibrated video footage.
O4: Risk-aware Operation for Safety and Efficiency
- Main Work: Incorporation of risk assessment into robotic decision-making, focusing on predicting and mitigating potential risks to ensure safety and operational efficiency.
- Key Results: Development of methodologies for anticipating navigation and manipulation risks, continuous monitoring (with causal reasoning algorithms) of risk levels during operations, and the implementation of a Stochastic Dynamic Programming engine (and task scheduler) to optimize decision-making under uncertainty. Development and integration of a Safe Motion Unit aligned with latest safety standards.
These achievements collectively enhance the capabilities of robotic systems in dynamic manipulation, human-robot interaction, autonomous navigation, and risk-aware operation, setting new standards for safety, efficiency, and technological innovation in the field of intralogistics robotics.
At the project end, two integrated mobile robotic platform have been developed and demonstrated to academic and industrial stakeholders, and have been showcased at a large open event (Automatica 2025, Munich). The system was capable of autonomously map the environment, interact efficiently with nearby humans, identify, grasp and throw objects to a dedicated target area. The consortium developed open-source libraries that can be used freely in the future. The project led to an italian patent, and extends the product portfolio of one of the industrial partners.
The DARKO project transcends existing boundaries in intralogistics robotics by pioneering the Bi-Stiffness Actuation (BSA) mechanism and algorithms for time- and energy-efficient dynamic manipulation, such as object throwing and precision pick-and-place actions.
This technical innovation enables robots to perform complex tasks with enhanced speed and reduced energy consumption. Additionally, DARKO advances the state of the art in human-robot interaction through sophisticated machine learning models for long-term human motion prediction and 3D human pose estimation, facilitating safer and more intuitive interactions between humans and machines in shared environments. The project's focus on autonomous system deployment leveraged cutting-edge techniques in self-supervised learning for semantic scene understanding and object detection, alongside novel approaches to failure-aware mapping and localization, significantly lowering the barrier to deploying intralogistics robots. By integrating these technologies, DARKO demonstrated a fully autonomous robotic system capable of navigating and operating in dynamic, real-world intralogistics settings, with a strong emphasis on risk-aware operation and safety. The outcomes include not only technological advancements but also contributions to the socio-economic landscape, such as increased production flexibility, cost savings, and the promotion of sustainable manufacturing practices through improved energy efficiency.
This technical innovation enables robots to perform complex tasks with enhanced speed and reduced energy consumption. Additionally, DARKO advances the state of the art in human-robot interaction through sophisticated machine learning models for long-term human motion prediction and 3D human pose estimation, facilitating safer and more intuitive interactions between humans and machines in shared environments. The project's focus on autonomous system deployment leveraged cutting-edge techniques in self-supervised learning for semantic scene understanding and object detection, alongside novel approaches to failure-aware mapping and localization, significantly lowering the barrier to deploying intralogistics robots. By integrating these technologies, DARKO demonstrated a fully autonomous robotic system capable of navigating and operating in dynamic, real-world intralogistics settings, with a strong emphasis on risk-aware operation and safety. The outcomes include not only technological advancements but also contributions to the socio-economic landscape, such as increased production flexibility, cost savings, and the promotion of sustainable manufacturing practices through improved energy efficiency.