Performance in Robots Interaction via Mental Imagery

Información del proyecto

PRIMI

Identificador del acuerdo de subvención: 101120727

DOI

10.3030/101120727

Fecha de la firma de la CE 3 Julio 2023

Fecha de inicio 1 Noviembre 2023

Fecha de finalización 31 Diciembre 2027

Financiado con arreglo a

• Digital, Industry and Space

Coste total

€ 4 973 683,75

Aportación de la UE

€ 4 973 683,75

Coordinado por

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Italy

Periodic Reporting for period 1 - PRIMI (Performance in Robots Interaction via Mental Imagery)

Período documentado: 2023-11-01 hasta 2024-10-31

Resumen del contexto y de los objetivos generales del proyecto

PRIMI aims to increase the adoption of personal social robots by addressing their current lack of autonomy and adaptability to individual behaviors and environmental conditions. The project will establish design principles for robots with human-like cognition and motor performance, merging various methodologies to create biologically plausible innovations. These innovations will enhance robots' self-awareness, environmental understanding, and Theory of Mind, enabling them to interact effectively with humans and adapt to dynamic situations. PRIMI's research will lead to advanced, socially aware humanoid robots, demonstrated through proof-of-principle prototypes like the neuromorphic iCub and the Kangaroo. These prototypes, based on different platforms, will showcase the modular and flexible solutions developed by PRIMI, paving the way for future personal robotics services. Indeed, the research programme will culminate in an impact case study in a clinical environment, where robots will act as tutors for the physical rehabilitation of patients. Clinicians will instruct the robots on mental practice and motor actions to replicate, prompting patient responses. The robots will monitor and autonomously adapt interactions to the patient's level, closing the multidisciplinary loop by using the same principles of action observation and motor mental simulations for both learning and rehabilitation.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

In the first reporting period, the PRIMI project has made significant progress across various work packages (WPs). WP1 focused on designing a bio-inspired cognitive architecture for neuromorphic systems, integrating mechanisms for efficient sensory processing and conducting studies to inform the design. WP2 developed a robot internal model for multimodal fusion and a dynamic planner for robot actions, inspired by the Mirror Neuron System. WP3 worked on abstract reasoning and theory of mind in robots. WP4 facilitated technology transfer between the iCub and Kangaroo robots, highlighting limitations of the REEM-C platform and prompting the decision to design and develop a new platform more suitable to take full advantage of PRIMI innovation. WP5 designed and built a PCB with SpiNNaker2 chips. WP6 upgraded the neuromorphic skin on the iCub, improved the MoveEnet Human Pose Estimation module, and developed a more efficient tactile sensory module. WP7 addressed ethical, legal, and social implications, while WP8 prepared a roadmap for the socio-economic opportunities of robotic technologies. These efforts collectively advance the project's goals and ensure compatibility with SpiNNaker2 hardware.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

Main Results:

Deliverable 4.1: This deliverable provides guidelines for transferring neuromorphic perception from the iCub robot to other humanoid robots. It includes an overview of the event-driven sensors in the iCub robot and proposes new perception architectures for transferring these sensors to other robots, such as the standard iCub in Manchester and the new Kangaroo robot by PAL Robotics. The architectures are designed to be flexible for installation on various humanoid robots.
Deliverable 5.1: This deliverable outlines the specifications for designing the SpiNNaker2 printed circuit board (PCB) for the PRIMI project. The SpiNNaker2 system will be integrated into the iCub and Kangaroo robots to explore brain-inspired machine learning algorithms for robotics. The PCB design includes parameters and requirements for connecting sensors and processing sensor data.
Deliverable 7.1: This report addresses the ethical, legal, and social implications (ELSI) of the technologies developed in PRIMI. It provides information on identifying, assessing, and managing ELSI aspects, including sustainability considerations. The PRIMI consortium aims to ensure these issues guide the research, development, and application of new technologies during and beyond the project.

Potential Impacts of the work done so far:

Enhanced Robot Perception: The integration of neuromorphic sensors and flexible perception architectures will improve the robots' ability to detect and respond to events, making them more adaptable and effective in various environments.
Advanced Cognitive Robotics: The SpiNNaker2 system's integration will enable the exploration of advanced brain-inspired machine learning algorithms, enhancing the cognitive capabilities of humanoid robots and their interactions with humans.
Ethical and Sustainable Development: Addressing ELSI aspects ensures that the development and application of new technologies are ethical, sustainable, and socially responsible, fostering public trust and acceptance of advanced robotics.

Project Logo