Skip to main content

Intentional stance for social attunement

Periodic Reporting for period 2 - InStance (Intentional stance for social attunement)

Reporting period: 2018-11-01 to 2020-04-30

The InStance project focuses on the question of whether (and under what conditions) people adopt an intentional mindset towards robots, a mindset that is typically adopted towards other humans. An intentional mindset is what the philosopher Daniel Dennett termed “intentional stance” - predicting and explaining behaviour with reference to the agent’s mental states such as beliefs, desires and intentions. To give an example: when I see a person gazing at a glass filled with water and extending their arm in its direction, I automatically surmise that the person intends to grasp it, because they feel thirst, believe that water will ease their thirst, and hence want to drink water from the glass. The terms “intend”, “feel” or “believe” all refer to mental states, and the assumption is that through referring to mental states, I can understand and explain someone else’s behaviour. However, for non-intentional systems (such as man-made artefacts), we often adopt the design stance - assuming that the system’s has been designed to behave in particular way (for example the car will slow down when one pushes the brakes not because the car intends to be slower, but because the car has been designed to slow down when the brake pedal is pushed).

Adopting either the intentional stance or the design stance is crucial not only for predicting others’ behaviour but presumably also for becoming engaged in a social interaction. That is, when I adopt the intentional stance, I direct my attention to where somebody is pointing, and hence we establish joint focus of attention, thereby becoming socially attuned. On the contrary, if I see that a machine’s artificial arm is pointing somewhere, I might be unwilling to attend there, as I do not believe that the machine wants to show me something, i.e. there is no intentional communicative content in the gesture.

This raises the question: to what extent are humans ready to adopt the intentional stance towards robots with human-like appearance, and to attune socially with them?

It might be that once a robot imitates human-like behaviour at the level of subtle (and often implicit) social signals, humans might automatically perceive its behaviour as reflecting mental states. This would presumably evoke social cognition mechanisms to the same (or similar) extent as in human-human interactions, allowing social attunement. By social attunement we mean a collection of mechanisms of social cognition that the brain employs during interactions with others: for example, joint attention, or visual-spatial perspective taking. Joint attention is a mechanism through which two or more individuals attend the same event or object in the environment. Engagement in joint attention often happens through directing others’ attention to where one is attending through, for example, gaze direction or a pointing gesture. Visual-spatial perspective taking is a mechanism that allows for taking someone else’s perspective in representation of space (for example, I understand that my “right” is “left” for my interaction partner, who is sitting opposite to me). In daily interactions with other humans we employ such mechanisms automatically. But would we employ similar mechanisms also in interaction with humanoid robots?

The objectives of the InStance project are to understand various factors that contribute to activating mechanisms of social attunement in interaction with humanoid robots, with a special focus on intentional stance. Does adopting intentional stance affect social attunement? What are the conditions for adopting the intentional stance, and social attunement?

In the InStance project we are specifically focusing on subtle characteristics of the robot behaviour, such as human-likeness of its movement parameters, mutual gaze, contingency of gaze behaviour or social intentions carried in communicative social signals. In addition, we are interested in how cultural embedding or famil
"In the first months of the InStance project, I established an entirely new lab, the Social Cognition in Human-Robot Interaction (S4HRI) lab, at the Italian Institute of Technology in Genova, Italy, where I decided to carry out my ERC project. The activities of S4HRI lab are to a large part linked with the InStance grant. I designed the lab space consisting of a sound-attenuated experimental cabin for human-robot interaction studies + EEG studies, and an external area with three experimental workstations for various psychophysical experiments. I have also equipped the lab with a humanoid robot iCub, with a Cozmo robot, a 64-channel EEG system and a mobile Tobii eye tracking system. More recently, I have also equipped the lab with a static Eyelink eyetracker and with another robot platform: the iCub robot head, mounted on a 3D-printed plastic body. This (half-)robot iCub is used for experiments where we use only robot head- and eye movements, and we do not need functionality of the entire robot body. This allows for avoiding bottlenecks in terms of robot availability for experiments.
Apart from equipping the lab, I recruited members of the InStance team: currently three PhD students, and two postdoctoral fellows. The members of InStance are assisted by other members of my team (supported from other funds): four postdoctoral fellows, one PhD student, and two software engineers.

The progress of the InStance project has been according to the plan. We have been addressing the following questions:
- Impact of human-like behavior of the robot on social attunement and adoption of intentional stance (work package 1)
- The role of communicative social signals (work package 2): here, we have addressed the issue of mutual gaze in social contexts, as well as action expectations and behavioural reciprocity (robot behavior contingent on the human behavior)
- Means and methods of probing adoption of the intentional stance (a topic transverse across all work packages)

(1) In order to address the question of how much human-like behaviour impacts social attunement, we first needed to collect data on parameters of behaviour in humans, in various tasks. To this end, we conducted a series of experiments in which we measured human eye and head movements that were to be implemented first on a robot simulator, and then on the embodied robot. We successfully managed to extract relevant features of human head and eye movements for producing human-like movements on the iCub robot. Our results showed that humans seem to rely more on temporal rather than spatial information when evaluating human-likeness of an observed behaviour. Interestingly, in terms of human sensitivity to human-like behaviour, we showed a discrepancy between objective measures (patterns of eye movements) and subjective reports. Specifically, participants attributed human-likeness to robot behaviours that were outside the human range, but they fixated longer on the behaviours that were within the human temporal range.
(2) To address questions related to communicative signals in social interaction, we conducted a series of experiments on mutual gaze with a robot in joint attention, and one experiment on mutual gaze with a robot in deception. The results showed that mutual gaze influences engagement in joint attention with a robot, and that it makes it more difficult for participants to provide false information to the robot, after the robot directs its gaze towards participants’ eyes. This shows that mutual gaze – being a potent social signal – has an impact on treating an artificial agent as a social entity, even though, strictly speaking, robot's ""gaze"" is just a pair of cameras. In addition to this series of studies, we conducted two experiments in which we examined the impact of contingent (reciprocal) behavior on social engagement with a robot. We showed that when the robot follows participants’ gaze direction, participants engage with the robot more, they like it more, a"
Although social attunement (in the sense of social cognition mechanisms such as joint attention or spatial perspective taking) has been widely investigated in the field of social neuroscience and psychology, specific factors influencing social attunement with various types of agents (artificial agents in particular) have not yet been systematically examined. It is not clear whether humans can socially attune to embodied artificial agents in real-time interactions, and what precisely are the behavioural characteristics of those agents that would allow for the attunement. InStance offers a novel avenue of research where various factors influencing social attunement are manipulated and their influence on fundamental mechanisms of social cognition is examined. In more detail, InStance uses real-time interaction protocols with humanoid robots and examines their behavioural characteristics (gaze behaviour, communicative signals) that have an impact on adopting the intentional stance and social attunement. Furthermore, in the next steps of the project, we will address higher-level factors that might play a role in adoption of intentional stance, factors such as cultural embedding or familiarity with robots. More specifically, we will address the question of whether Asian cultures differ from Western cultures in the likelihood of adopting intentional stance towards artefacts, and if so, what are the key elements involved in such cultural differences.

InStance’s results shall allow for understanding whether certain behavioural parameters of the robot behavior (or certain other factors) are more crucial than others regarding engagement of the human user in social interaction with a robot. Moreover, InStance’s findings will provide a set of guidelines for roboticists regarding robot behaviours that should facilitate social attunement. In addition, we plan, as one of InStance’s outcomes, a test battery that would probe adoption of the intentional stance towards other agents, with combined explicit and implicit measures. Further, we aim at identifying not only behavioural but also neural markers of the intentional stance. Finally, the findings of InStance might explain to what extent cultural background, individual differences and experience shape attitudes towards humanoid robots.

In summary, InStance offers a unique and interdisciplinary contribution to the fields of social neuroscience and engineering (social robotics in particular), as it aims to systematically investigate specific parameters of an agent’s behaviour that allow for attribution of mental states, and thereby social attunement. In addition, it is linked to philosophy through addressing (and operationalising) the philosophical concept of the intentional stance.

The area of social and cognitive neuroscience should benefit from InStance by gaining knowledge about what aspects of human behaviour make us being perceived as intentional agents whose actions appear to result from mental operations. This is a fundamental question related to social cognition in social interactions, because being perceived as an intentional agent allows for establishing a common social context with others. Such issues are addressed in fundamental research on social cognition and also in philosophy. InStance offers a unique approach to study this question, due to the use of embodied presence of a complex artificial agent (humanoid robot) allowing for human-like natural social interactions on the one hand, and experimental control on the other.

In the context of applied areas of engineering, InStance is aiming to help in designing machines that are acceptable by human users at the most fundamental level of implicit cognitive processes. Although InStance’s focus is on complex humanoid robots, as they offer embodied human-like presence, findings of inStance might become generalisable to other devices that interact with humans (e.g. virtual personal assistants on mobile phones, interfac
Illustration of InStance methods: combining interaction protocol with cognitive neuroscience methods
Example experiment: manipulating reciprocity (contingency) in joint attention
Experimental setup: Participant seated with iCub in the cabin, experimenter monitors EEG signal