Periodic Reporting for period 1 - NEUROBUKIMI (Affective neurocognitive model for the ‘Uncanny Valley’)
Reporting period: 2016-08-01 to 2018-07-31
Ongoing advances in computer graphics and robotics technologies have allowed the development of increasingly humanlike robots and computer-animated virtual characters for consumer, service, health care, entertainment, and research purposes. Such developments may come with a cost, however. The uncanny valley hypothesis (UVH) predicts that virtual characters or robots that resemble real humans too closely can elicit feelings of unease, eeriness, and lack of familiarity. Despite its importance, the validity of UVH has not yet been firmly established. The UV phenomenon is often taken for granted; for example, specific animation films are often anecdotally cited as exemplars of the uncanny valley. In contrast, empirical research has still provided inconsistent evidence for this phenomenon.
A deeper understanding of UVH is important for society for two reasons: first, it can guide the development of human-like robots and virtual characters and second, it can help us predict how individuals will react to them. Various kinds of socially interactive robots have already begun to appear for consumer, service, and health care applications. Two examples include seal-like robot PARO that is being used for therapeutic purposes and ROBEAR robot that is being used in nursing care. Purposefully human-like robots are also being developed by companies such as Hanson Robotics. In entertainment, the first attempts to create fully animated human-like cinematic actors were already made shortly after the turn of the present millennium when films such as Beowulf and The Polar Express saw their premieres. Such films met an overall negative critical reception that focused on the “uncanniness” of their highly realistic animations, and the UVH was even explicitly mentioned in some reviews. In science, virtual characters are now being used increasingly more often to replace research stimuli recorded from real humans. This is the case for example for neurocognitive experiments on affects and social cognition, in which faces and facial expressions have long served as research stimuli. Because the UVH makes a general recommendation to avoid high levels of realism and human-likeness, it is of direct relevance to all of the aforementioned applications.
This project approached the UVH mainly from the perspective of social-affective neurocognitive research; however, its results were expected to be of more general societal relevance as well. We focused mainly on computer-generated (CG) faces given the importance of faces in social interaction and the wide-spread use of CG faces in neurocognitive studies. The overall objectives of this project included assessing whether and when the UV exists and investigating the similarities and differences in the neurocognitive processing of real and CG faces.
We include the following conclusions for the action (for explanations, see the section below): i) uncanny valley phenomenon exists but it is much weaker than usually thought and strongly influenced by social factors; ii) contrary to UVH, more realistic CG faces are actually more desirable; iii) CG faces elicit different cognitive, affective, and neural processing than real faces.
A deeper understanding of UVH is important for society for two reasons: first, it can guide the development of human-like robots and virtual characters and second, it can help us predict how individuals will react to them. Various kinds of socially interactive robots have already begun to appear for consumer, service, and health care applications. Two examples include seal-like robot PARO that is being used for therapeutic purposes and ROBEAR robot that is being used in nursing care. Purposefully human-like robots are also being developed by companies such as Hanson Robotics. In entertainment, the first attempts to create fully animated human-like cinematic actors were already made shortly after the turn of the present millennium when films such as Beowulf and The Polar Express saw their premieres. Such films met an overall negative critical reception that focused on the “uncanniness” of their highly realistic animations, and the UVH was even explicitly mentioned in some reviews. In science, virtual characters are now being used increasingly more often to replace research stimuli recorded from real humans. This is the case for example for neurocognitive experiments on affects and social cognition, in which faces and facial expressions have long served as research stimuli. Because the UVH makes a general recommendation to avoid high levels of realism and human-likeness, it is of direct relevance to all of the aforementioned applications.
This project approached the UVH mainly from the perspective of social-affective neurocognitive research; however, its results were expected to be of more general societal relevance as well. We focused mainly on computer-generated (CG) faces given the importance of faces in social interaction and the wide-spread use of CG faces in neurocognitive studies. The overall objectives of this project included assessing whether and when the UV exists and investigating the similarities and differences in the neurocognitive processing of real and CG faces.
We include the following conclusions for the action (for explanations, see the section below): i) uncanny valley phenomenon exists but it is much weaker than usually thought and strongly influenced by social factors; ii) contrary to UVH, more realistic CG faces are actually more desirable; iii) CG faces elicit different cognitive, affective, and neural processing than real faces.
The work performed in this project has addressed the following questions: i) does the UV exist in animation films, ii) does the UV exist in rigorously controlled face images, iii) do real and CG faces perform similarly in a face memory task, and iv) do CG faces with varying gaze directions and facial expressions elicit similar neural activations as real faces, as measured with functional magnetic resonance imaging (fMRI).
As an overview, the results show that i) there is evidence of the UV in animated film characters but this effect is much weaker than anecdotally claimed and it can also be modulated by exposure to critical film reviews; ii) there is a strong trend between higher realism and familiarity (more realistic faces elicit more favorable evaluations) and this effect shadows a weak UV effect occurring specifically for CG faces; iii) CG faces perform differently from real faces in a face memory task such that CG faces elicit a greater proportion of false recollections (the “they all look the same to me” phenomenon); and iv) CG faces trigger weaker fMRI responses in a face-sensitive brain network than real faces.
Results have been disseminated to the scientific community (and to some extent, industry) in four scientific conferences, via scientific publications, and via a guest lecture given at the course “Man-Machine Interaction” at Maastricht University (March 20th 2017); and to the general public via an interview published in New Scientist (October 29th 2016).
As an overview, the results show that i) there is evidence of the UV in animated film characters but this effect is much weaker than anecdotally claimed and it can also be modulated by exposure to critical film reviews; ii) there is a strong trend between higher realism and familiarity (more realistic faces elicit more favorable evaluations) and this effect shadows a weak UV effect occurring specifically for CG faces; iii) CG faces perform differently from real faces in a face memory task such that CG faces elicit a greater proportion of false recollections (the “they all look the same to me” phenomenon); and iv) CG faces trigger weaker fMRI responses in a face-sensitive brain network than real faces.
Results have been disseminated to the scientific community (and to some extent, industry) in four scientific conferences, via scientific publications, and via a guest lecture given at the course “Man-Machine Interaction” at Maastricht University (March 20th 2017); and to the general public via an interview published in New Scientist (October 29th 2016).
The present project pushes the state of the art in UV research by demonstrating that the UV phenomenon is of rather limited significance, at least when it comes to static face images typically employed in empirical research. Even though CG faces were found to elicit a weak UV effect, this effect was negligible in comparison to a straightforward relationship between realism and familiarity. That is, the results supported an “uncanny slope” rather than an uncanny valley: more realistic faces simply elicited more favorable reactions in a linear manner. Future research is still required to validate these findings in more realistic stimuli such as anthropomorphic robots. Nevertheless, the present findings already have some economical and societal implications for all applications in consumer, service, health care, and entertainment sectors that plan to employ human-like virtual characters. The present findings encourage the development of highly realistic human-like entities, and they should also alleviate any concerns over the uncanny valley in such applications.
On the other hand, the present findings also emphasize the limits of virtual characters by showing that virtual as compared with realistic faces still trigger different cognitive and affective responses both at behavioral and neural levels. Notably, these differences were observed using static images of faces –even greater differences should be expected for moving and interactive artificial entities. The present findings are of significance for research (social-affective neuroscience in particular) but also for the wider societal context. The findings suggest that in order to reach identical neurocognitive processing as real faces, virtual characters need to be made practically identical to them. Bridging the gap between real and virtual characters remains a formidable challenge especially for fully-embodied, moving, and socially interactive characters.
On the other hand, the present findings also emphasize the limits of virtual characters by showing that virtual as compared with realistic faces still trigger different cognitive and affective responses both at behavioral and neural levels. Notably, these differences were observed using static images of faces –even greater differences should be expected for moving and interactive artificial entities. The present findings are of significance for research (social-affective neuroscience in particular) but also for the wider societal context. The findings suggest that in order to reach identical neurocognitive processing as real faces, virtual characters need to be made practically identical to them. Bridging the gap between real and virtual characters remains a formidable challenge especially for fully-embodied, moving, and socially interactive characters.