The missing link between Perception and Cognition: The case of multiple-person scenarios

Project Information

THEMPO

Grant agreement ID: 758473

DOI

10.3030/758473

Project closed

EC signature date 21 December 2017

Start date 1 March 2018

End date 31 August 2024

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 499 375,00

EU contribution

€ 1 499 375,00

Coordinated by

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE CNRS
France

Periodic Reporting for period 4 - THEMPO (The missing link between Perception and Cognition: The case of multiple-person scenarios)

Reporting period: 2022-09-01 to 2024-08-31

Summary of the context and overall objectives of the project

The study of the visual system on an ontogenetic and phylogenetic time scale, has shown that biologically relevant redundant features of the visual world may drive specific perceptual adaptations, calling for efficient coding in dedicated structures of the visual cortex. Given that the core aspect of social life is relationship between two or more entities, we reasoned that human vision might have developed efficient coding of perceptual units larger and more complex than single social entities. The mechanisms discovered in THEMPO account for how the human brain achieves complex social tasks with extreme efficiency, and reveal uncharted functions of human vision. The implications of THEMPO are characterized in relation to three fields: vision, cognitive neuroscience and clinical neuroscience.
First, our research on visual perception is contributing to address the goals and boundaries of human vision. Our findings show that visual perception goes beyond segmentation and recognition of single objects, processing with high efficiency and specialization the relations between them, which give rise to representations of scenes and events.
Second, the capacity to form and manipulate relational representations is considered a fundamental, possibly unique, aspect of human intelligence. THEMPO seeks to explain how the mind/brain transforms visual input into object representations and objects representations into structured representations of scenes or events, where the relation with the others specifies the role of each part.
Finally, perceptual tasks involving socially relevant stimuli have contributed to highlight differences in social sensitivity between individuals with pervasive developmental disorders such as autism spectrum disorders (ASD) and the normal population. We address this relationship using the newly defined category of multiple-person relations. Showing that performance in our visual perception tasks varies with the inter-individual variation in social cognitive abilities is a critical test-bed for the hypothesis that perception concerns the social domain. Moreover, our experimental stimuli and tasks can provide new tools for assessment in ASD, offering performance-based measures of the individual’s abilities, without requiring explicit reports, judgments or verbal instructions. This represents a significant advantage for assessment of individuals with communication difficulties. Moreover, the tasks developed in THEMPO have been adapted to study the development of visual discrimination based visuo-spatial relations between people, in subjects as young as 6-month-olds. Thus, our research can meet the joint effort of cognitive and clinical neuropsychologists to develop tasks that can capture early alterations in basic perceptual functions relevant for social cognition.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Our current research findings are revealing a number of behavioral and neural phenomena that suggest the existence of early perceptual adaptations for processing multiple-body scenarios –i.e. configurations of multiple bodies in specific spatial relations that cue social interaction (e.g. two face-to-face bodies). We have shown that vision encodes visuo-spatial relations between bodies with the same efficiency and high specialization of face/body perception. Specifically, perception of face-to-face (vs. non-facing) bodies evokes effects compatible with the most robust markers of face-specificity such as the behavioral inversion effect and increased activity in selective visual areas. Moreover, face-to-face bodies are processed as a grouped unit, analogously to facial features in a face. These signatures of domain-specificity in visual processing suggest a remarkably early effect of social parameters in the visual system. Our research on very young infants (from 5-month-olds) further shows early emergence of perceptual functions that may ground or affect the (later) development of social cognitive abilities.
We have published four papers (Papeo, Goupil, Soto-Faraco, 2020 ; Papeo, Abassi, 2019; Abassi & Papeo, 2020; Papeo, 2020) and three are under review (Adibpour, Hochmann & Papeo, 2020; Goupil, Papeo & Hochmann, 2020; Bellot, Abassi & Papeo, 2020).
Dissemination through participation to scientific events:

L. Papeo has organized the Symposium “Seeing Relations: From Multiple-Object Perception to the Representation of Social Events”, at the International Convention of Psychological Science 2019, Paris.
Other contributions:
Talks
L. Papeo (selected): Vision Science Society conference 2019; Language Lab, Massachusetts Institute of Technology, Boston, May 2019; Department of Psychology, Harvard University, May 2019; the Science of Social Interaction Workshop, Bangor University, UK, Sept 2018
E. Abassi: Virtual Vision Science Society Conference 2020

Posters:
FENS 2020 Virtual forum, July 11-15, 2020¬ (E. Bellot: poster AWARDED)
Virtual Vision Science Society Conference 2020 (C. Spriet)
International Congress for Infant Studies Virtual Congress 2020 (C. Spriet)
Organization for the Human Brain Mapping 2019 (P. Adibpour)
European Conference on Visual Perception 2019 (P. Adibpour)
International Convention of Psychological Science, 2019 (E. Abassi)
Concepts Actions and Objects Workshop, Rovereto, Italy, May 2018 (2018)

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Our findings show that human vision cares about objects as well as relations between objects. In particular, multiple –seemingly interacting– entities form single units of attention, processed with the highest priority and specialization in human vision. This research is revealing uncharted functions of vision and the building blocks of social cognition.
Future developments of THEMPO concern three main issues. First, we aim at understanding the mechanisms underlying the representation of social interaction in visual areas, in relation to other brain networks involved in the representation social interaction (e.g. action understanding or theory-of-mind networks). Second, we use transcranial magnetic stimulation to establish brain-behavior causal relationship, by testing the functional relevance of the brain areas responsive during perception of interacting bodies. Third, our research provides strong indications of domain specificity in the visual system and its precocious appearance in development. We will address the emergence of the visual specialization for two-body stimuli in the context of the larger functional organization of the human visual system. By studying young infants, we study how the ability to process visuo-spatial relations between multiple objects and to use this information for discrimination of complex visual scenes relates to the development of object categorization in human vision.

Markers of visual specificity to multiple-body configurations