Periodic Reporting for period 1 - TeAMH-Robot (Temporal Adaptation and anticipation Mechanisms in Human-Robot interaction) Reporting period: 2021-08-01 to 2022-07-31 Summary of the context and overall objectives of the project Shortly, the advent of robots will change the role that artificial agents play in our life, as the interaction with them will not be limited to specialized and well-structured work environments. When interacting with artificial agents, humans often occur into the Out of The Loop phenomenon (OOTL), which is a difficulty to predict and prevent failures, probably due to a lack of transparency in artificial agents' actions. The overarching aim of TeAMH-Robot is to develop a model of Human-Robot Interaction (HRI) that will prevent the OOTL phenomenon when interacting with robots. To this end, TeAMH-Robot will use a novel approach to develop an efficient HRI that combines cognitive neuroscience methods with real-time interactive tasks with a humanoid robot. Firstly, TeAMH-Robot will aim to understand how temporal adaptation and anticipation mechanisms allow humans to “stay in the loop” when mistakes occur. Then, TeAMH-Robot will use this knowledge to develop a human-inspired module for humanoid robots that ensure real-time coordination. Finally, TeAMH-Robot will test the efficiency of human-inspired behaviour in reducing the impact of the OOTL phenomenon. The outgoing phase was planned to be hosted by Western Sydney University (Australia), where the ER ran activities aiming to (i) identify and (ii) model temporal adaptation and anticipation mechanisms that allow humans to predict, prevent, and recover mistakes. The return phase is hosted by the Italian Institute of Technology (Italy), where the ER will (iii) develop a module for humanoid robots that ensure real-time coordination and (iv) test its efficiency in reducing the OOTL using the acquired neurocognitive methods. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The scientific objectives of the outgoing phase were to identify temporal adaptation and anticipation mechanisms in human-human interactions characterized by errors. This objective seeks to answer the question: What are the mechanisms that allow humans to stay “inside the loop” when interacting with other human agents? A key outcome was supposed to identify how and to which extent temporal adaptation and anticipation mechanisms are crucial to ensure the co-agent’s performance monitoring and its successful prediction of and recovery from errors. To this end, the ER has designed and implemented two experiments using musical paradigms to study sensorimotor synchronization. Experiment 1 aims to evaluate adaptation and anticipations in the asynchrony during a leader-follower musical task characterized by errors. Specifically, pair of participants performed a coordination task in which one member of the pair acted as a leader, i.e. s/he has access to the visual metronome, and s/he was instructed to play a melody trying to synchronize with a visual computerized metronome. The metronome was programmed to deviate from the correct melody by generating errors in content but not in timing. The second member of the pair, i.e. the follower, was instructed to play the correct melody synchronizing in timing (but not in content) with the leader. Data of 22 pairs have been collected. Data analysis of Experiment 1 is ongoing. Experiment 2 aims to evaluate neural correlates of attention allocation towards errors during a musical task in which human participants synchronize with a computerized metronome that makes both content and timing errors. Participants were presented with a visual metronome presenting the timing of a sequence to play. During the task, participants were asked to passively observe the metronome and to play the sequence trying to synchronize with the visual metronome presented on the screen. In each session the metronome executed an error in content, i.e. it mismatches one element of the sequence with another one. Across blocks, we manipulated the timing of the events preceding the error. Specifically, the metronome was programmed to be constant or variable in the timing of events before the content error occurred. Behavioural and EEG data of 30 participants have been collected. Data analysis of Experiment 2 is ongoing.TeAMH-Robot also aimed to increase the independency of the ER and enrich her expertise in transferable skills, such as grant writing, teaching, dissemination, and outreach. The ER has co-supervised both Ph.D. and Master's Degree candidates, and she held a series of departmental seminars across different universities in Europe and overseas (i.e. Italy, Denmark, and Australia). She attended international conferences both on the topic of cognitive and social neuroscience and robotics. She organized an interactive demonstration of TeAMH-Robot at a Science Fair targeting pupils and adults. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) To date, research on coordination in HRI has been traditionally limited to emergent coordination that arises spontaneously and the main objective of these studies was to evaluate whether and to which extent humans synchronize with a robotic agent. In this way, the robot has been used as an embodied metronome. However, most of HRIs in the real world will occur in applied contexts and will be characterized by planned and goal-driven behaviours. At the same time, research on planned coordination between human agents has been mainly focused on temporal adaptation and anticipation mechanisms in successful social interactions, i.e. when no errors occur. This prevented the understanding of how temporal adaptation and anticipation mechanisms allowed to prevent and recover errors. At the current stage, TeAMH-Robot is significantly extending the state of the art in the field of cognitive and social neuroscience. By analysing temporal adaptation and anticipation mechanisms during erring social interactions, it is possible to extend computational models of human-human interactions to situations in which errors are likely to occur. TeAMH-Robot will extend the state of the art in the field of robotics. By developing a behavioural model for humanoid robots based on temporal adaptation and anticipation of the human co-agent to promote effective HRI, TeAMH-Robot will reduce the probability that the OOTL phenomena will occur and ensure the safe and smooth integration of robots in society. An example of the experimental set up applied in TeAMH-Robot
