Periodic Reporting for period 2 - PERSEO (European Training Network on PErsonalized Robotics as SErvice Oriented applications)
Período documentado: 2023-01-01 hasta 2024-12-31
The ETN on PErsonalized Robotics as SErvice Oriented applications - PERSEO - aims to train a new generation of research and professional figures to face the research challenges of the market of personal robots. The personal robotics domain faces challenges in personalizing robot behavior to individual needs, requiring deeper analysis by companies and the development of an interdisciplinary research community in academia to enable adaptive, cost-effective, and easily configurable robotic systems.
The development of personalizable robots is accelerating. However, only a small amount of such robots is deployed in society, tackling very specific tasks in well-controlled environments. In PERSEO three scenarios will serve as integration milestones to assess the Early Stage Researchers (ESR) research progression and to evaluate the benefit of the service-oriented approach for the integration of the research projects: Personal Robots as Companions for Elderly People, Personal Robots for Physical Rehabilitation and Assistance, and Personal Robots for Edutainment.
PERSEO's research program is divided into three themes focusing on robot personalization at the "Physical," "Cognitive," and "Social" levels, requiring expertise across fields like computer science, AI, automation, ethics, and psychology. ESRs developed specialized knowledge through individual research projects, interdisciplinary training, and secondments, while also addressing the social, legal, and ethical challenges of personal robots to ensure alignment with European values.
The development of personalizable robots is accelerating. However, only a small amount of such robots is deployed in society, tackling very specific tasks in well-controlled environments. In PERSEO three scenarios will serve as integration milestones to assess the Early Stage Researchers (ESR) research progression and to evaluate the benefit of the service-oriented approach for the integration of the research projects: Personal Robots as Companions for Elderly People, Personal Robots for Physical Rehabilitation and Assistance, and Personal Robots for Edutainment.
PERSEO's research program is divided into three themes focusing on robot personalization at the "Physical," "Cognitive," and "Social" levels, requiring expertise across fields like computer science, AI, automation, ethics, and psychology. ESRs developed specialized knowledge through individual research projects, interdisciplinary training, and secondments, while also addressing the social, legal, and ethical challenges of personal robots to ensure alignment with European values.
15 Early-Stage Researchers were recruited. The ESRs participated in several cross-sectorial environments within and outside the consortium in Industrial, Academic and public engagement events. During the first PERSEO residential workshops, they lectured on technical aspects related to personalization as well as on law, psychology, and management.
Network-wide events were organized in the PERSEO project during the whole duration of the program. The PERSEO project organizes four main kinds of training events: (1) Residential Workshops; (2) Winter and Summer Schools; (3) International Workshops and (4) Public Dissemination events.
Network-wide events were organized in the PERSEO project during the whole duration of the program. The PERSEO project organizes four main kinds of training events: (1) Residential Workshops; (2) Winter and Summer Schools; (3) International Workshops and (4) Public Dissemination events.
To create personalisation and adaptation in robots, the technical work in the project has been divided into the 3 main WPs, aimed at defining the role of personalization at different levels of possible HRI:
WP3 - Personalization in Physical Interaction:
● ESR 2 Object and People Recognition for Personalized Interaction, enhancing HRI by developing robust models for action recognition, behavior prediction, and sequence reconstruction, enabling robots to interpret human actions and improve engagement in real-world tasks like education, rehabilitation, and assistance.
● ESR 3 Interactive Kinaesthetic Teaching for Conditional Tasks, enhancing HRI by developing and evaluating kinesthetic teaching approaches, enabling non-roboticists to teach robots new tasks while assessing their impact on trust, psychological ownership, and applications in edutainment and occupational therapy.
● ESR 4 Human Action Learning and Physical Capabilities Modelling, advancing spatially-grounded semantics for robots by developing a saliency-based framework, creating reMap for spatial and symbolic data representation, and leveraging Large Language Models to enhance intuitive and transparent human-robot interactions.
● ESR 5 Adaptive Robot Movements for Transparent Interaction, using transparency in HRI by incorporating non-verbal cues, learning-based adaptations, and Theory of Mind, while also developing TOROS, a standardized scale for measuring robot transparency.
WP4 - Personalization in Cognitive Interaction:
● ESR 6 Human intention recognition and prediction for human-robot collaborative behaviours, advancing intention recognition by integrating low-level motions with high-level semantic activity, enabling context-based human activity interpretation, real-time robot adaptation, and improved collaborative task execution through high-level reasoning and imitation learning.
● ESR 7 Integration of vision and language for human-robot interaction, developing algorithms for language-based instruction, integrating vision and natural language to enable robots to recognize actions, understand their environment, describe their state, follow instructions, and engage in empathetic dialogue for enhanced cognitive HRI.
● ESR 8 Theory of Mind for Personalized Interaction, using Theory of Mind for robots to infer human mental states, focusing on desires and beliefs, and demonstrated a strong correlation between ToM and transparency, enhancing user interactions and validating its role in improving collaborative human-robot interaction.
● ESR 9 Modelling User Cognition for Adaptive HRI, focusing on user cognitive modeling to adapt robot behavior and enhance intuitive human-robot interfaces by developing a general model of user preferences, integrating AI and machine learning, and leveraging neuromorphic computing for real-time cognitive load assessment.
● ESR 10 Robot as Schoolmate for Enhanced Adaptive Learning, developing personalized, adaptive robots that assist teachers by inferring children's cognitive and emotional states, promoting a comfortable, guided play learning environment, though real-time modeling of personality and emotions remained challenging.
WP5 - Personalization in Social Interaction:
● ESR 11 Deep Learning for Adaptive Human-Robot Interaction. The ESR11 project developed deep learning models for adaptive human-robot interaction, focusing on recognizing human emotions through vocal bursts, multiparty conversation cues, and facial expressions, with an emphasis on personalization and continual learning to improve emotion recognition in HRI.
● ESR 12 Personalized Social Cues for Robots as Information Providing Interfaces. ESR12's research focused on personalized social cues in robots, developing the Spontaneous Interaction State Machine to model non-verbal behaviors like gaze and emotional expressions, and creating a modular engagement metric to enhance robot interactions and engagement in dynamic social settings.
● ESR 13 Personalization and Trust toward Robots. ESR13's research explored how personalization and customization in robot adaptation influence trust, proposing that robot autonomy fosters anthropomorphization and user involvement promotes psychological ownership, both of which positively impact trust in robots.
● ESR 14 Deception. ESR14's research developed a theoretical framework for regulating HRI, focusing on robot design, psychological effects, and legal implications, to address manipulation risks and create a balanced approach for user protection and technological innovation.
● ESR 15 Ethical aspects of personalization in companion and care robots. ESR15's project explored the ethics of social robots through art and personalisation, examining issues like trust, safety, transparency, and biases, while using interdisciplinary research to enhance the design of safer, more user-friendly robots.
The PERSEO consortium implemented a comprehensive dissemination strategy, engaging scientific communities through international conferences and workshops, while reaching the general public via social media, fairs, events, and press releases.
WP3 - Personalization in Physical Interaction:
● ESR 2 Object and People Recognition for Personalized Interaction, enhancing HRI by developing robust models for action recognition, behavior prediction, and sequence reconstruction, enabling robots to interpret human actions and improve engagement in real-world tasks like education, rehabilitation, and assistance.
● ESR 3 Interactive Kinaesthetic Teaching for Conditional Tasks, enhancing HRI by developing and evaluating kinesthetic teaching approaches, enabling non-roboticists to teach robots new tasks while assessing their impact on trust, psychological ownership, and applications in edutainment and occupational therapy.
● ESR 4 Human Action Learning and Physical Capabilities Modelling, advancing spatially-grounded semantics for robots by developing a saliency-based framework, creating reMap for spatial and symbolic data representation, and leveraging Large Language Models to enhance intuitive and transparent human-robot interactions.
● ESR 5 Adaptive Robot Movements for Transparent Interaction, using transparency in HRI by incorporating non-verbal cues, learning-based adaptations, and Theory of Mind, while also developing TOROS, a standardized scale for measuring robot transparency.
WP4 - Personalization in Cognitive Interaction:
● ESR 6 Human intention recognition and prediction for human-robot collaborative behaviours, advancing intention recognition by integrating low-level motions with high-level semantic activity, enabling context-based human activity interpretation, real-time robot adaptation, and improved collaborative task execution through high-level reasoning and imitation learning.
● ESR 7 Integration of vision and language for human-robot interaction, developing algorithms for language-based instruction, integrating vision and natural language to enable robots to recognize actions, understand their environment, describe their state, follow instructions, and engage in empathetic dialogue for enhanced cognitive HRI.
● ESR 8 Theory of Mind for Personalized Interaction, using Theory of Mind for robots to infer human mental states, focusing on desires and beliefs, and demonstrated a strong correlation between ToM and transparency, enhancing user interactions and validating its role in improving collaborative human-robot interaction.
● ESR 9 Modelling User Cognition for Adaptive HRI, focusing on user cognitive modeling to adapt robot behavior and enhance intuitive human-robot interfaces by developing a general model of user preferences, integrating AI and machine learning, and leveraging neuromorphic computing for real-time cognitive load assessment.
● ESR 10 Robot as Schoolmate for Enhanced Adaptive Learning, developing personalized, adaptive robots that assist teachers by inferring children's cognitive and emotional states, promoting a comfortable, guided play learning environment, though real-time modeling of personality and emotions remained challenging.
WP5 - Personalization in Social Interaction:
● ESR 11 Deep Learning for Adaptive Human-Robot Interaction. The ESR11 project developed deep learning models for adaptive human-robot interaction, focusing on recognizing human emotions through vocal bursts, multiparty conversation cues, and facial expressions, with an emphasis on personalization and continual learning to improve emotion recognition in HRI.
● ESR 12 Personalized Social Cues for Robots as Information Providing Interfaces. ESR12's research focused on personalized social cues in robots, developing the Spontaneous Interaction State Machine to model non-verbal behaviors like gaze and emotional expressions, and creating a modular engagement metric to enhance robot interactions and engagement in dynamic social settings.
● ESR 13 Personalization and Trust toward Robots. ESR13's research explored how personalization and customization in robot adaptation influence trust, proposing that robot autonomy fosters anthropomorphization and user involvement promotes psychological ownership, both of which positively impact trust in robots.
● ESR 14 Deception. ESR14's research developed a theoretical framework for regulating HRI, focusing on robot design, psychological effects, and legal implications, to address manipulation risks and create a balanced approach for user protection and technological innovation.
● ESR 15 Ethical aspects of personalization in companion and care robots. ESR15's project explored the ethics of social robots through art and personalisation, examining issues like trust, safety, transparency, and biases, while using interdisciplinary research to enhance the design of safer, more user-friendly robots.
The PERSEO consortium implemented a comprehensive dissemination strategy, engaging scientific communities through international conferences and workshops, while reaching the general public via social media, fairs, events, and press releases.