To create personalisation and adaptation in robots, the technical work in the project has been divided into the 3 main WPs, aimed at defining the role of personalization at different levels of possible HRI:
WP3 - Personalization in Physical Interaction:
● ESR 2 Object and People Recognition for Personalized Interaction, enhancing HRI by developing robust models for action recognition, behavior prediction, and sequence reconstruction, enabling robots to interpret human actions and improve engagement in real-world tasks like education, rehabilitation, and assistance.
● ESR 3 Interactive Kinaesthetic Teaching for Conditional Tasks, enhancing HRI by developing and evaluating kinesthetic teaching approaches, enabling non-roboticists to teach robots new tasks while assessing their impact on trust, psychological ownership, and applications in edutainment and occupational therapy.
● ESR 4 Human Action Learning and Physical Capabilities Modelling, advancing spatially-grounded semantics for robots by developing a saliency-based framework, creating reMap for spatial and symbolic data representation, and leveraging Large Language Models to enhance intuitive and transparent human-robot interactions.
● ESR 5 Adaptive Robot Movements for Transparent Interaction, using transparency in HRI by incorporating non-verbal cues, learning-based adaptations, and Theory of Mind, while also developing TOROS, a standardized scale for measuring robot transparency.
WP4 - Personalization in Cognitive Interaction:
● ESR 6 Human intention recognition and prediction for human-robot collaborative behaviours, advancing intention recognition by integrating low-level motions with high-level semantic activity, enabling context-based human activity interpretation, real-time robot adaptation, and improved collaborative task execution through high-level reasoning and imitation learning.
● ESR 7 Integration of vision and language for human-robot interaction, developing algorithms for language-based instruction, integrating vision and natural language to enable robots to recognize actions, understand their environment, describe their state, follow instructions, and engage in empathetic dialogue for enhanced cognitive HRI.
● ESR 8 Theory of Mind for Personalized Interaction, using Theory of Mind for robots to infer human mental states, focusing on desires and beliefs, and demonstrated a strong correlation between ToM and transparency, enhancing user interactions and validating its role in improving collaborative human-robot interaction.
● ESR 9 Modelling User Cognition for Adaptive HRI, focusing on user cognitive modeling to adapt robot behavior and enhance intuitive human-robot interfaces by developing a general model of user preferences, integrating AI and machine learning, and leveraging neuromorphic computing for real-time cognitive load assessment.
● ESR 10 Robot as Schoolmate for Enhanced Adaptive Learning, developing personalized, adaptive robots that assist teachers by inferring children's cognitive and emotional states, promoting a comfortable, guided play learning environment, though real-time modeling of personality and emotions remained challenging.
WP5 - Personalization in Social Interaction:
● ESR 11 Deep Learning for Adaptive Human-Robot Interaction. The ESR11 project developed deep learning models for adaptive human-robot interaction, focusing on recognizing human emotions through vocal bursts, multiparty conversation cues, and facial expressions, with an emphasis on personalization and continual learning to improve emotion recognition in HRI.
● ESR 12 Personalized Social Cues for Robots as Information Providing Interfaces. ESR12's research focused on personalized social cues in robots, developing the Spontaneous Interaction State Machine to model non-verbal behaviors like gaze and emotional expressions, and creating a modular engagement metric to enhance robot interactions and engagement in dynamic social settings.
● ESR 13 Personalization and Trust toward Robots. ESR13's research explored how personalization and customization in robot adaptation influence trust, proposing that robot autonomy fosters anthropomorphization and user involvement promotes psychological ownership, both of which positively impact trust in robots.
● ESR 14 Deception. ESR14's research developed a theoretical framework for regulating HRI, focusing on robot design, psychological effects, and legal implications, to address manipulation risks and create a balanced approach for user protection and technological innovation.
● ESR 15 Ethical aspects of personalization in companion and care robots. ESR15's project explored the ethics of social robots through art and personalisation, examining issues like trust, safety, transparency, and biases, while using interdisciplinary research to enhance the design of safer, more user-friendly robots.
The PERSEO consortium implemented a comprehensive dissemination strategy, engaging scientific communities through international conferences and workshops, while reaching the general public via social media, fairs, events, and press releases.
