investigating Human Shared PErception with Robots

Project Information

wHiSPER

Grant agreement ID: 804388

DOI

10.3030/804388

Project closed

EC signature date 6 November 2018

Start date 1 March 2019

End date 31 December 2024

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 749 375,00

EU contribution

€ 1 749 375,00

Coordinated by

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Italy

Periodic Reporting for period 4 - wHiSPER (investigating Human Shared PErception with Robots)

Reporting period: 2023-09-01 to 2024-12-31

Summary of the context and overall objectives of the project

Perception is a dynamic process influenced by prior experience, shaping how individuals interpret sensory stimuli. The wHiSPER project investigated how these mechanisms of perceptual inference are modulated in social interactions, particularly when engaging with both human and robotic partners. By integrating psychophysical methods, Bayesian modeling, and advanced humanoid robotics, wHiSPER provided a novel framework for understanding how perception adapts in collaborative settings.
The project demonstrated for the first time that interaction with a robot perceived as a social agent alters both spatial and temporal perception, highlighting that human perceptual processes extend beyond individual stability to accommodate shared experiences. Experiments revealed that engagement in joint activities leads individuals to modify their perception, aligning it with their partner’s. This adaptation was shown to occur in both spatial perception and temporal perception.
Another key achievement was the development of computational models and robotic architectures capable of replicating human-like perceptual inference. In addition to reinforcement learning and continual adaptation strategies, wHiSPER developed Bayesian models that allow robots to predict and understand perceptual distortions in human partners. This capability enabled the robotic agents not only to dynamically refine their own perception but also to anticipate how humans might perceive spatial and temporal properties in a shared environment, contributing to the broader goal of creating socially intelligent robotic systems capable of establishing shared perception with human partners.
wHiSPER’s findings have implications for various domains requiring precise coordination, including rehabilitation, collaborative robotics, and human-centered artificial intelligence. More broadly, the project provided insights into how to develop robotic partners and sensorized devices that can assess and adapt to individual perceptual differences, improving human interaction in both assistive and professional settings. These findings pave the way for new technologies that support individuals with sensory and motor impairments, enhancing perception and coordination through adaptive artificial systems.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The wHiSPER project investigated how human perception is influenced by social interaction, particularly in settings involving both human and robotic agents. The project leveraged psychophysical experimentation, computational modeling, and advanced robotics to study how perceptual inference adapts during interaction, with a strong emphasis on developing robotic systems capable of shared perception.
Throughout the project, wHiSPER advanced five primary objectives:
1. Understanding Perceptual Adaptation in Interaction – The project demonstrated that human perception of both spatial and temporal properties shifts in interactive contexts, even with a robot, emphasizing the role of social engagement in perceptual inference.
2. Assessing How Perceptual Priors Generalize to the Observation of Others' Actions – The project studied how prior experience influences action perception, particularly in relation to movement style. This effect extended to robotic actions, demonstrating that humans apply the same perceptual models when interacting with artificial agents.
3. Developing Computational Models of Shared Perception – wHiSPER developed Bayesian models to quantify how perceptual inference adapts during social interaction. These models provided insights into how individuals dynamically balance internal priors with external social cues.
4. Endowing Robots with Shared Perception Abilities – The project explored reinforcement learning and continual adaptation strategies to enhance robotic perception and action coordination. Novel robotic architectures were developed, allowing robots to adapt to human behavior in real time, improving their capacity for natural and intuitive interaction.
5. Investigating Perceptual Adaptation Across Different Sensory Conditions – A key development during the project was the realization that perceptual adaptation mechanisms vary depending on individual sensory characteristics. Research extended to populations with sensory impairments and aging-related perceptual changes, highlighting how prior experience influences perception differently across diverse conditions.

The project resulted in a substantial body of interdisciplinary publications. In engineering and robotics, findings were published in journals such as IEEE Transactions on Cognitive and Developmental Systems, IEEE Access, and the International Journal of Social Robotics. Contributions to neuroscience appeared in Cerebral Cortex and Frontiers in Human Neuroscience, while psychology-related results were featured in Frontiers in Psychology and Experimental Psychology: General. High-impact multidisciplinary outlets, including PNAS and Scientific Reports, further demonstrated the project's broad relevance.
wHiSPER findings were also presented at leading international conferences, earning Best Paper Awards and Honorable Mentions. Beyond academia, the project gained media attention across television, radio, and online platforms, highlighting its significance for both the scientific community and the general public.
Furthermore, the methodologies developed have potential applications in assistive robotics, rehabilitation, and human-centered AI, offering novel solutions for technology-mediated social interaction.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

wHiSPER has significantly advanced the understanding of perceptual inference in social contexts. The project further redefined how artificial systems should be designed, moving beyond conventional AI approaches toward Artificial Cognition, an embodied and proactive model of intelligence that enables dynamic and adaptive interaction with humans.
Key contributions beyond the state of the art include:
1. Empirical Demonstration of Interactive Perceptual Inference – The project was the first to show that perceptual mechanisms such as regression to the mean, a fundamental aspect of perception, are modulated by social interaction. This finding has implications for understanding how humans achieve perceptual alignment in joint activities.
2. Integration of Robotics and Psychophysics – wHiSPER introduced a new methodological approach, using humanoid robots as experimental probes to systematically manipulate social interaction dynamics. This allowed for a level of control and reproducibility that is difficult to achieve in human-human studies.
3. Development of Adaptive Robotic Systems – The project provided computational frameworks for robots to dynamically adjust their perception and actions based on human partners. These advancements contribute to the design of next-generation socially adaptive robots, capable of engaging in fluid and context-sensitive interactions.
Moving forward, the insights gained from wHiSPER will inform the development of robotic systems capable of adapting their perception and actions in real-time, enhancing their effectiveness in collaborative settings such as healthcare, education, and assistive technology. Additionally, the methodologies developed in this project could be applied to investigate perceptual adaptation in individuals with neurocognitive disorders, offering potential diagnostic and therapeutic applications.

Banner