Performance in Robots Interaction via Mental Imagery

Projektinformationen

PRIMI

ID Finanzhilfevereinbarung: 101120727

DOI

10.3030/101120727

EK-Unterschriftsdatum 3 Juli 2023

Startdatum 1 November 2023

Enddatum 31 Dezember 2027

Finanziert unter

• Digital, Industry and Space

Gesamtkosten

€ 4 973 683,75

EU-Beitrag

€ 4 973 683,75

Koordiniert durch

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Italy

Periodic Reporting for period 1 - PRIMI (Performance in Robots Interaction via Mental Imagery)

Berichtszeitraum: 2023-11-01 bis 2024-10-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

PRIMI aims to increase the adoption of personal social robots by addressing their current lack of autonomy and adaptability to individual behaviors and environmental conditions. The project will establish design principles for robots with human-like cognition and motor performance, merging various methodologies to create biologically plausible innovations. These innovations will enhance robots' self-awareness, environmental understanding, and Theory of Mind, enabling them to interact effectively with humans and adapt to dynamic situations. PRIMI's research will lead to advanced, socially aware humanoid robots, demonstrated through proof-of-principle prototypes like the neuromorphic iCub and the Kangaroo. These prototypes, based on different platforms, will showcase the modular and flexible solutions developed by PRIMI, paving the way for future personal robotics services. Indeed, the research programme will culminate in an impact case study in a clinical environment, where robots will act as tutors for the physical rehabilitation of patients. Clinicians will instruct the robots on mental practice and motor actions to replicate, prompting patient responses. The robots will monitor and autonomously adapt interactions to the patient's level, closing the multidisciplinary loop by using the same principles of action observation and motor mental simulations for both learning and rehabilitation.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

In the first reporting period, the PRIMI project has made significant progress across various work packages (WPs). WP1 focused on designing a bio-inspired cognitive architecture for neuromorphic systems, integrating mechanisms for efficient sensory processing and conducting studies to inform the design. WP2 developed a robot internal model for multimodal fusion and a dynamic planner for robot actions, inspired by the Mirror Neuron System. WP3 worked on abstract reasoning and theory of mind in robots. WP4 facilitated technology transfer between the iCub and Kangaroo robots, highlighting limitations of the REEM-C platform and prompting the decision to design and develop a new platform more suitable to take full advantage of PRIMI innovation. WP5 designed and built a PCB with SpiNNaker2 chips. WP6 upgraded the neuromorphic skin on the iCub, improved the MoveEnet Human Pose Estimation module, and developed a more efficient tactile sensory module. WP7 addressed ethical, legal, and social implications, while WP8 prepared a roadmap for the socio-economic opportunities of robotic technologies. These efforts collectively advance the project's goals and ensure compatibility with SpiNNaker2 hardware.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

Main Results:

Deliverable 4.1: This deliverable provides guidelines for transferring neuromorphic perception from the iCub robot to other humanoid robots. It includes an overview of the event-driven sensors in the iCub robot and proposes new perception architectures for transferring these sensors to other robots, such as the standard iCub in Manchester and the new Kangaroo robot by PAL Robotics. The architectures are designed to be flexible for installation on various humanoid robots.
Deliverable 5.1: This deliverable outlines the specifications for designing the SpiNNaker2 printed circuit board (PCB) for the PRIMI project. The SpiNNaker2 system will be integrated into the iCub and Kangaroo robots to explore brain-inspired machine learning algorithms for robotics. The PCB design includes parameters and requirements for connecting sensors and processing sensor data.
Deliverable 7.1: This report addresses the ethical, legal, and social implications (ELSI) of the technologies developed in PRIMI. It provides information on identifying, assessing, and managing ELSI aspects, including sustainability considerations. The PRIMI consortium aims to ensure these issues guide the research, development, and application of new technologies during and beyond the project.

Potential Impacts of the work done so far:

Enhanced Robot Perception: The integration of neuromorphic sensors and flexible perception architectures will improve the robots' ability to detect and respond to events, making them more adaptable and effective in various environments.
Advanced Cognitive Robotics: The SpiNNaker2 system's integration will enable the exploration of advanced brain-inspired machine learning algorithms, enhancing the cognitive capabilities of humanoid robots and their interactions with humans.
Ethical and Sustainable Development: Addressing ELSI aspects ensures that the development and application of new technologies are ethical, sustainable, and socially responsible, fostering public trust and acceptance of advanced robotics.

Project Logo