Neural markers of social brain activation in human-robot interaction

Project Information

NeuroMarkerHRI

Grant agreement ID: 101063504

DOI

10.3030/101063504

Project closed

EC signature date 22 July 2022

Start date 1 June 2023

End date 30 April 2026

Funded under

• Marie Skłodowska-Curie Actions (MSCA)

Total cost

No data

EU contribution

€ 184 077,36

Coordinated by

TECHNISCHE UNIVERSITAT BERLIN
Germany

Periodic Reporting for period 1 - NeuroMarkerHRI (Neural markers of social brain activation in human-robot interaction)

Reporting period: 2023-06-01 to 2026-04-30

Summary of the context and overall objectives of the project

Robots are increasingly moving out of factories and into everyday environments such as hospitals, schools and homes, where they work alongside people rather than performing isolated tasks. A fundamental question remains open: does the human brain treat a robot as a social partner, the way it treats another person, or simply as a sophisticated tool? The answer shapes how robots should be designed, how much we can trust them, and how safely humans and robots can collaborate.
The human brain has dedicated mechanisms for understanding others' intentions and mental states, known as social cognition, which allow us to anticipate behaviour and coordinate with one another. Robots with human-like appearance and movement can create the impression that they, too, have intentions, even though their behaviour is ultimately programmed. Whether this impression engages the brain's genuine social mechanisms, or instead relies on more general cognitive processes such as attention, was not well understood. In particular, this is crucial during real physical interaction rather than passive observation on a screen.
NeuroMarkerHRI addressed this gap by combining cognitive neuroscience, robotics and data science. Using a safe, portable neuroimaging method that allows brain activity to be measured during real physical interaction with a robot, the project pursued three objectives: determining whether interacting with a humanoid robot recruits the same brain regions involved in understanding other people's minds; identifying which aspects of robot behaviour influence this engagement; and using machine learning to translate brain activity patterns into a practical marker of social versus tool-like engagement with a robot.
The project also examined what builds and restores trust between people and artificial agents, a question of direct relevance to the responsible design of robots and AI systems in collaborative and high-stakes settings such as healthcare and industry. By combining neuroscience, robotics and machine learning, NeuroMarkerHRI aimed to set new methodological standards for studying the social brain in real-world human-robot interaction and to provide a scientific foundation for human-centred robot design.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The project combined behavioural experiments, neuroimaging and machine learning to study how the human brain responds to interacting with a humanoid robot. The original screen-based approach was adapted into real, physical interaction studies. This required substantial technical development to integrate behavioural measurement software with the robot's control systems.
The core experimental task was a validated risk-taking paradigm, adapted for participants to perform together with a humanoid robot under collaborative or competitive conditions. In collaborative conditions, participants took risks at levels comparable to human-human collaboration, suggesting the robot was engaged with as a genuine social partner. In competitive conditions, participants behaved more strategically, frequently outperforming the robot. These findings were published in a peer-reviewed paper at a top-tier conference (the ACM/IEEE International Conference on Human-Robot Interaction) and are available as an open-access preprint.
A second study examined whether giving the robot a distinct personality, varying in expressiveness and responsiveness, influenced trust during collaboration. Personality had no measurable effect on trust or performance. Instead, only the robot's actual behavioural reliability predicted trust, an unexpected but practically important finding for robot design.
The central neuroimaging phase involved collecting brain activity data using a high-density, portable method while participants engaged in the same collaborative and competitive tasks, including both interaction and observation conditions. A substantial dataset was successfully collected from a gender-balanced group of participants, targeting brain regions associated with understanding others' intentions and mental states. Analysis of this dataset, combined with machine learning to identify patterns that distinguish social from non-social processing, is in its final stages, with results expected shortly after the project ends.
Related studies investigated how trust in artificial agents forms, breaks down after errors, and can be repaired, testing strategies such as apologies, warmth and shared activity with a robot. A consistent pattern emerged, suggesting that communicative repair strategies provided little benefit, while actual reliability remained the primary driver of trust recovery. A further study using electroencephalography found that collaborating with an artificial agent heightened people's awareness of their own mistakes, without changing how they processed the agent's errors.
Overall achievements include a validated real-interaction paradigm, a substantial neuroimaging dataset, a peer-reviewed publication at a top-tier venue, and converging evidence that behavioural reliability outweighs social signalling in shaping trust in robots and artificial agents. The neuroimaging and machine learning analyses, the project's most technically ambitious objective, are nearing completion and are expected to yield the first validated neural markers of social engagement with humanoid robots during real interaction.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

NeuroMarkerHRI advances the state of the art as one of the first projects to study the human brain's response to robots during real, physical, collaborative interaction, rather than passive screen-based observation. Combining functional near-infrared spectroscopy (fNIRS), a high-density, portable neuroimaging method, with a validated behavioural paradigm and machine learning analysis represents a methodological step forward for the field, offering a model that other researchers can adopt to study social cognition in authentic human-robot interaction settings.
The project's behavioural findings go beyond prior work by demonstrating, with a directly comparable paradigm, that people's risk-taking behaviour with a robot mirrors human-human collaboration when the interaction is framed cooperatively, while shifting towards strategic, goal-directed behaviour in competitive settings. This nuanced, context-dependent picture moves beyond earlier studies that treated human-robot interaction as a single, uniform category.
A further contribution beyond the current state of the art is the consistent, converging evidence across multiple independent studies that behavioural reliability, rather than communicative or expressive social signalling such as personality, warmth or apologies, is the primary driver of trust in robots and artificial agents. This challenges common assumptions in robot and AI system design, which often prioritise anthropomorphic or communicative features as trust-building tools.
The neuroimaging and machine learning component, once complete, is expected to deliver the first validated, non-invasive neural marker that indicates whether a person is engaging with a robot as a social partner or as a tool during real interaction. Such a marker would represent a genuinely novel scientific and technological contribution, with no directly comparable tool currently available to the field.
Several key needs remain to ensure full uptake and impact of these results. Completion and peer-reviewed publication of the neuroimaging analysis is the immediate priority, as this is necessary to validate the proposed neural marker and make it usable by the wider research and robotics community. Further research is needed to test the marker's generalisability across different robot platforms, interaction contexts and populations, beyond the single robot platform and laboratory setting used in this project. Translating the findings on trust and reliability into concrete design guidelines would benefit from closer engagement with robotics companies and standardisation bodies working on human-robot interaction and trustworthy AI. Finally, continued institutional support and funding, including the collaborations and follow-up funding applications initiated during the project, will be essential to bring the neuroimaging findings and best-practice protocol to publication and to ensure their uptake by the broader scientific and industrial community.