Cracking the neural code of human object vision

Información del proyecto

CRACK

Identificador del acuerdo de subvención: 803370

DOI

10.3030/803370

Proyecto cerrado

Fecha de la firma de la CE 11 Diciembre 2018

Fecha de inicio 1 Mayo 2019

Fecha de finalización 31 Enero 2025

Financiado con arreglo a

• EXCELLENT SCIENCE - European Research Council (ERC)

Coste total

€ 1 480 576,00

Aportación de la UE

€ 1 480 576,00

Coordinado por

FREIE UNIVERSITAET BERLIN
Germany

Periodic Reporting for period 4 - CRACK (Cracking the neural code of human object vision)

Período documentado: 2023-11-01 hasta 2025-01-31

Resumen del contexto y de los objetivos generales del proyecto

With every blink of our eyes, our brain effortlessly distills out of the stream of photons hitting the retina a conscious percept of the world. This percept consists of enclosed and meaningful entities – objects – and determines how we interact with the world. Over the course of about one hundred years, psychology and neuroscience have identified the core part of the brain mediating object vision. Understanding human object recognition is important for several reasons. It can give an answer to why do we perceive the way we do. Understanding the mechanisms also bears promise in clinical application, e.g. restitution of vision after brain damage. It is also of strong interest to industry and society as the biological brain remains the most exciting inspiration for building artificial vision systems. Yet, the mechanisms that mediate the human ability to recognize objects remain incompletely understood.
The overall goal of CRACK was to provide answers to three fundamental, long-standing and open questions about visual object recognition that build on each other: 1) How does each of the core cortical regions active during vision represent objects, 2) how do those regions communicate information, and 3) how does the observed dynamic neural activity mediate adaptive behavior? The first objective was to unravel the unique importance and role of each core cortical region of the visual brain involved in object recognition. We have controlled, isolated, and further described the nature of representations in visual cortex. The second objective was to clarify how those core cortical regions communicate with each other. Using layer-specific fMRI and time-frequency resolved EEG we have described feedforward and recurrent communication streams. The third objective was to link the observed mechanisms to behavior. We related neural activity to behavior using multiple methods, highlighting the aspects of visual representations suitably formatted to support behavior. Fulfilling these objectives through an orchestrated and interdisciplinary effort CRACK provides key empirical pieces of evidence for an updated theory of visual object recognition in the human cortex.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

We have now addressed all three objectives. Concerning objective 1, we have a) developed and applied a computational approach to probe the function of human brain regions, b) conducted a series of experiments detailing how object representations are influenced by the typical, structured scene environments in which they appear, b) shown how fundamental object properties category and location are represented and d) investigated the representation of animacy. Concerning objective 2, we have disentangled information flow in feedforward and feedback direction a) in the spatial domain across cortical layers, and b) in the temporal domain in oscillatory channels. Concerning objective 3, we have identified visual representations suitably formatted to guide behaviour using several techniques.

The results of this project were and further will be disseminated through peer-reviewed articles, onference presentations, tutorials, in invited talks, and through the development of an online-prediction challenge called the Algonauts Project. The results have further significantly contributed to the development of several new research directions that are being continued by previous team members.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

Our work goes beyond the state-of-the-art in several aspects. For example, the computational approach elucidating the functions of visual brain regions from a new algorithmic perspective that previous approaches did not do to comparable depth. Second, our work on object category and location representations invites revision of the standard theory of visual processing. Third, a challenge we conducted proposes a new way of how to do science in a way that ensures efficiency and openness in cognitive neuroscience. Fourth, our work on how visual regions communicate demonstrate the methodology to resolve feedforward and feedback information flow in both space and in time and provide novel insight into the mechanisms of communication in both domains. Fifth, we provided a large neuroimaging data set for video that we analyzed using novel methods, highlighting in the temporal dimension of visual experience neglected typically in object recognition research.

erc-image.png