Periodic Reporting for period 3 - InStance (Intentional stance for social attunement)
Reporting period: 2020-05-01 to 2021-10-31
Adopting either the intentional stance or the design stance is crucial not only for predicting others’ behaviour but presumably also for becoming engaged in a social interaction. That is, when I adopt the intentional stance, I direct my attention to where somebody is pointing, and hence we establish joint focus of attention, thereby becoming socially attuned. On the contrary, if I see that a machine’s artificial arm is pointing somewhere, I might be unwilling to attend there, as I do not believe that the machine wants to show me something, i.e. there is no intentional communicative content in the gesture.
This raises the question: to what extent are humans ready to adopt the intentional stance towards robots with human-like appearance, and to attune socially with them?
It might be that once a robot imitates human-like behaviour at the level of subtle (and often implicit) social signals, humans might automatically perceive its behaviour as reflecting mental states. This would presumably evoke social cognition mechanisms to the same (or similar) extent as in human-human interactions, allowing social attunement. By social attunement we mean a collection of mechanisms of social cognition that the brain employs during interactions with others: for example, joint attention, or visual-spatial perspective taking. Joint attention is a mechanism through which two or more individuals attend the same event or object in the environment. Engagement in joint attention often happens through directing others’ attention to where one is attending through, for example, gaze direction or a pointing gesture. Visual-spatial perspective taking is a mechanism that allows for taking someone else’s perspective in representation of space (for example, I understand that my “right” is “left” for my interaction partner, who is sitting opposite to me). In daily interactions with other humans we employ such mechanisms automatically. But would we employ similar mechanisms also in interaction with humanoid robots?
Apart from equipping the lab, I recruited members of the InStance team: currently three PhD students, and two postdoctoral fellows. The members of InStance are assisted by other members of my team (supported from other funds): four postdoctoral fellows, one PhD student, and two software engineers.
The progress of the InStance project has been according to the plan. We have been addressing the following questions:
- Impact of human-like behavior of the robot on social attunement and adoption of intentional stance (work package 1)
- The role of communicative social signals (work package 2): here, we have addressed the issue of mutual gaze in social contexts, as well as action expectations and behavioural reciprocity (robot behavior contingent on the human behavior)
- Means and methods of probing adoption of the intentional stance (a topic transverse across all work packages)
(1) In order to address the question of how much human-like behaviour impacts social attunement, we first needed to collect data on parameters of behaviour in humans, in various tasks. To this end, we conducted a series of experiments in which we measured human eye and head movements that were to be implemented first on a robot simulator, and then on the embodied robot. We successfully managed to extract relevant features of human head and eye movements for producing human-like movements on the iCub robot. Our results showed that humans seem to rely more on temporal rather than spatial information when evaluating human-likeness of an observed behaviour. Interestingly, in terms of human sensitivity to human-like behaviour, we showed a discrepancy between objective measures (patterns of eye movements) and subjective reports. Specifically, participants attributed human-likeness to robot behaviours that were outside the human range, but they fixated longer on the behaviours that were within the human temporal range.
(2) To address questions related to communicative signals in social interaction, we conducted a series of experiments on mutual gaze with a robot in joint attention, and one experiment on mutual gaze with a robot in deception. The results showed that mutual gaze influences engagement in joint attention with a robot, and that it makes it more difficult for participants to provide false information to the robot, after the robot directs its gaze towards participants’ eyes. This shows that mutual gaze – being a potent social signal – has an impact on treating an artificial agent as a social entity, even though, strictly speaking, robot's "gaze" is just a pair of cameras. In addition to this series of studies, we conducted two experiments in which we examined the impact of contingent (reciprocal) behavior on social engagement with a robot. We showed that when the robot follows participants’ gaze direction, participants engage with the robot more, they like it more, and they perceive it more as human-like. We also addressed the question of gaze as communicative signal in action expectations. We showed that violations of expectations regarding robot gaze in the context of action sequence have an impact on engagement in joint attention.
(3) Finally, to address the issue measuring adoption of the intentional stance, we developed a questionnaire in which series of pictures are presented to participants, depicting a short “story”. In the questionnaire, participants are asked to choose between a mentalistic and more mechanistic interpretation of what is happening in the depicted story. Results of a first study in which we administered the questionnaire online showed that in some contexts, participants selected the mentalistic over the mechanistic interpretation, suggesting that in principle it is possible that intentional stance is adopted towards a robot. In follow-up experiments, we have observed individual differences in likelihood of adopting the intentional stance towards robots, not only in terms of decision scores but also in patterns of the EEG signal. Interestingly, we also observed a negative correlation between the ISQ and years of education, meaning that the more years of education a participant has, the less likely it is for him/her to adopt intentional stance towards a robot. In addition to developing an explicit measure of adoption of intentional stance, we are aiming to also find more implicit measures. One measure is the phenomenon of reduced sense of agency when performing tasks with others – a phenomenon observed in social interactions with other humans. We reasoned that if such an effect can also be observed in human-robot interaction, this would mean that participants treat the robot as an intentional agent. We designed two experiments, in which we investigated Sense of Agency (SoA) in interaction with artificial agents. Results showed reduced SoA when participants were jointly responsible for the action with a robot, but not with a mechanical device, suggesting that participants treated the robot as an intentional agent. Furthermore, we also showed an effect of vicarious sense of agency (measured as a so-called vicarious intentional binding effect). Intentional binding is a phenomenon observed when participants are asked to judge the time when an event occurs. It has been shown that when a sensory event occurs as a result of one's voluntary (intentional) action, the estimate of the time when the action was generated appears closer in time to the sensory effect (typically a tone), and/or the time of the occurrence of the sensory event is judged as closer to the action event, thereby producing the intentional binding effect. Such effect has also been reported for experiments in which participants observed others performing the task. Interestingly, in our studies, we show this effect also when participants observe a robot performing the task, suggesting that the robot might be treated as an intentional agent.
In summary, the first half of the InStance project have been concluded with:
- several important insights on likelihood of adopting the intentional stance towards robots and on factors contributing to social attunement with robots,
- implementation of some of human-like behaviour on the humanoid robot,
- tools for probing the intentional stance.
We have shown that humanoid robots have the potential to produce behaviours that are treated as social signals (e.g. mutual gaze contact or gaze reciprocity) and that they might evoke attribution of intentional agency, as documented by the vicarious sense of agency effect, and the reduced sense of agency for one's own action outcomes.
InStance’s results shall allow for understanding whether certain behavioural parameters of the robot behavior (or certain other factors) are more crucial than others regarding engagement of the human user in social interaction with a robot. Moreover, InStance’s findings will provide a set of guidelines for roboticists regarding robot behaviours that should facilitate social attunement. In addition, we plan, as one of InStance’s outcomes, a test battery that would probe adoption of the intentional stance towards other agents, with combined explicit and implicit measures. Further, we aim at identifying not only behavioural but also neural markers of the intentional stance. Finally, the findings of InStance might explain to what extent cultural background, individual differences and experience shape attitudes towards humanoid robots.
In summary, InStance offers a unique and interdisciplinary contribution to the fields of social neuroscience and engineering (social robotics in particular), as it aims to systematically investigate specific parameters of an agent’s behaviour that allow for attribution of mental states, and thereby social attunement. In addition, it is linked to philosophy through addressing (and operationalising) the philosophical concept of the intentional stance.
The area of social and cognitive neuroscience should benefit from InStance by gaining knowledge about what aspects of human behaviour make us being perceived as intentional agents whose actions appear to result from mental operations. This is a fundamental question related to social cognition in social interactions, because being perceived as an intentional agent allows for establishing a common social context with others. Such issues are addressed in fundamental research on social cognition and also in philosophy. InStance offers a unique approach to study this question, due to the use of embodied presence of a complex artificial agent (humanoid robot) allowing for human-like natural social interactions on the one hand, and experimental control on the other.
In the context of applied areas of engineering, InStance is aiming to help in designing machines that are acceptable by human users at the most fundamental level of implicit cognitive processes. Although InStance’s focus is on complex humanoid robots, as they offer embodied human-like presence, findings of inStance might become generalisable to other devices that interact with humans (e.g. virtual personal assistants on mobile phones, interfaces of autonomous driving vehicles). In terms of the specific field of social robotics, InSance will provide information on characteristics of robot behaviour that elicit social attunement. This might be particularly useful for healthcare applications of assistive robotics. The field of social robotics is a promptly developing area with high societal impact, as it aims at designing robots that provide care in hospitals, rehabilitation centers, programs/therapies for children (e.g. children diagnosed with autism spectrum disorder) and the elderly. Often, however, the methods assessing human engagement in an interaction with a robot are based on questionnaires, which cannot measure implicit mechanisms of social cognition; and are vulnerable to social desirability biases. It is thus crucial for social robotics to develop robots based on research with objective methods, such as the approach of InStance, for fundamentally social human-robot interaction.