AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics

Projektinformationen

IntelliMan

ID Finanzhilfevereinbarung: 101070136

DOI

10.3030/101070136

Projekt abgeschlossen

EK-Unterschriftsdatum 13 Juni 2022

Startdatum 1 September 2022

Enddatum 30 April 2026

Finanziert unter

• Digital, Industry and Space

Gesamtkosten

€ 4 509 303,00

EU-Beitrag

€ 4 509 303,00

Koordiniert durch

ALMA MATER STUDIORUM - UNIVERSITA DI BOLOGNA
Italy

Periodic Reporting for period 3 - IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics)

Berichtszeitraum: 2025-01-01 bis 2026-04-30

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

Recent advances in Artificial Intelligence, robot perception, and autonomous manipulation have expanded the capabilities of robotic systems in service, industrial, and assistive applications. Nevertheless, enabling robots to autonomously acquire manipulation skills, transfer knowledge across heterogeneous platforms, and safely operate in complex and dynamic environments remains a major scientific and technological challenge.
The IntelliMan project addresses these challenges by developing an AI-powered manipulation system capable of persistent learning, adaptive decision-making, and human-aware interaction. The project combines multimodal perception, machine learning, shared autonomy, tactile sensing, and task planning to allow robots to perceive their environment, autonomously execute manipulation tasks, detect failures, and continuously improve their performance through interaction with humans and the environment.
A distinctive feature of IntelliMan is its modular and cross-platform approach, allowing algorithms and manipulation capabilities to be transferred across different robotic embodiments and application domains. The developed technologies have been validated in four representative use cases covering upper-limb prosthetics, household service robotics, flexible manufacturing involving deformable objects, and fresh food handling for logistics. This cross-domain validation demonstrates the portability of perception, control, learning, and shared-autonomy modules while reducing the effort required to adapt robotic systems to new tasks.
Beyond technological innovation, IntelliMan investigates the human factors associated with AI-powered robotic manipulation, including user trust, embodiment, explainability, and shared autonomy. Through the integration of adaptive control, sensory feedback, knowledge representation, and human-centred evaluation, the project contributes to the development of intelligent robotic systems that are safer, more reliable, and better accepted in real-world applications.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

Completed the final integration and validation of AI-powered manipulation systems across four representative use cases: upper-limb prosthetics, household service robotics, flexible manufacturing, and fresh food handling;
Developed modular perception, control, tactile sensing, and shared-autonomy components that were successfully transferred across multiple robotic platforms and application scenarios;
Integrated synthetic-data-based object pose estimation pipelines for robust perception in household, industrial, and food-handling environments, reducing the need for manually labelled training data;
Implemented ontology-based planning, adaptive Behaviour Trees, PDDL reasoning, and knowledge-grounding methods for autonomous task execution in household manipulation;
Developed reinforcement-learning-based task refinement pipelines capable of adapting demonstrated manipulation skills to new tasks through policy optimisation and simulation;
Integrated multimodal tactile sensing, force monitoring, and vibrotactile feedback into prosthetic hands, service robots, and industrial manipulation systems to improve grasp robustness and human interaction;
Developed incremental and co-adaptive myocontrol strategies based on sEMG signals and shared-autonomy mechanisms for intuitive prosthetic control and industrial robot teaching;
Validated stereo vision, 6D pose estimation, and dual-arm robotic manipulation for deformable wire insertion and connector assembly in industrial environments;
Developed simulation-to-real grasping policies, compliant multi-fingered grippers, RGB-D perception, and tactile feedback for autonomous fresh food handling and packing operations;
Established KPI-based evaluation frameworks to assess manipulation performance across all demonstrators;
Consolidated dissemination, exploitation, and technology transfer activities, including multiple exploitable results, patent applications, open-source software components, and industrial validation campaigns.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

UNIBO, FAU and INAIL developed an integrated shared-autonomy framework for upper-limb prostheses combining incremental myocontrol, HMM-based autonomy arbitration, tactile sensing and vibrotactile feedback, improving grasp regulation, user embodiment and intuitive control.
UPC introduced an ontology-driven knowledge representation and task planning framework integrating Behaviour Trees, semantic reasoning and high-level command grounding, enabling robots to autonomously interpret and execute complex household manipulation tasks.
EUT, DLR and UPC developed an AI-based task refinement pipeline that combines human demonstrations, reinforcement learning and Large Language Model-assisted reward shaping, achieving successful transfer of manipulation policies to new tasks with minimal additional demonstrations.
UNIBO and UCLV developed advanced stereo vision and 6D pose estimation systems for precise manipulation of deformable linear objects, enabling autonomous connector assembly with dual-arm robotic platforms in industrial environments.
UNIBO, ELVEZ and UCLV integrated wearable sEMG interfaces, vibrotactile feedback and human-in-the-loop supervision into flexible manufacturing, allowing intuitive robot teaching and adaptive execution of complex assembly operations.
OCADO, DLR and project partners demonstrated simulation-to-real grasping strategies for fresh food handling, combining compliant multi-fingered grippers, tactile sensing and reinforcement learning to achieve high picking accuracy while handling delicate products.
IDIAP introduced novel ergodic control and active sensing approaches together with reusable distance-field representations for manipulation planning, improving environmental exploration and algorithm portability across robotic platforms.
The project demonstrated extensive cross-platform portability of perception, learning, tactile sensing and shared-autonomy modules across prosthetic systems, service robots, and industrial manipulators, validating IntelliMan's modular AI-based manipulation framework in four distinct application domains.