Skip to main content

Cracking the neural code of human object vision

Periodic Reporting for period 2 - CRACK (Cracking the neural code of human object vision)

Reporting period: 2020-11-01 to 2022-04-30

With every blink of our eyes, our brain effortlessly distills out of the stream of photons hitting the retina a conscious percept of the world. This percept consists of enclosed and meaningful entities – objects – and determines how we interact with the world. Over the course of about one hundred years, psychology and neuroscience have identified the core part of the brain mediating object vision. Understanding human object recognition is important for several reasons. It can give an answer to why do we perceive the way we do. Understanding the mechanisms also bears promise in clinical application, e.g. restitution of vision after brain damage. It is also of strong interest to industry and society as the biological brain remains the most exciting inspiration for building artificial vision systems. Yet, the mechanisms that mediate the human ability to recognize objects remain incompletely understood.
The overall goal of CRACK is to provide answers to three fundamental, long-standing and open questions about visual object recognition that build on each other: 1) How does each of the core cortical regions active during vision represent objects, 2) how do those regions communicate information, and 3) how does the observed dynamic neural activity mediate adaptive behavior? The first objective is to unravel the unique importance and role of each core cortical region of the visual brain involved in object recognition. The second objective is to clarify how those core cortical regions communicate with each other. The third objective is to link the observed mechanisms to behavior. Fulfilling these objectives through an orchestrated and interdisciplinary effort CRACK aims to provide the empirical pieces of evidence for an updated theory of visual object recognition in the human cortex.
Since the beginning of the project we have made progress in spite of challenging circumstances caused by the covid-19 pandemic. We have focused on addressing objective 1 and have made most progress there, we are right in the middle of addressing objective 2, and are taking first steps towards addressing objective 3. Concerning objective 1, we have a) developed and applied a computational approach to probe the function of human brain regions, b) conducted a series of experiments detailing how object representations are influenced by the typical, structured scene environments in which they appear, b) shown how fundamental object properties category and location are represented and d) investigated the representation of animacy. Concerning objective 2, we have disentangled information flow in feedforward and feedback direction a) in the spatial domain across cortical layers, and b) in the temporal domain in oscillatory channels. Concerning objective 3, we have shown how behavior ratings of object animacy relate to the representation of animacy in the brain.
Our work goes beyond the state-of-the-art in several aspects. For example, the computational approach elucidating the functions of visual brain regions from a new algorithmic perspective that previous approaches did not do to comparable depth. Second, our work on object category and location representations invites revision of the standard theory of visual processing. Third, a challenge we conducted proposes a new way of how to do science in a way that ensures efficiency and openness in cognitive neuroscience. Fourth, our work on how visual regions communicate demonstrate the methodology to resolve feedforward and feedback information flow in both space and in time and provide novel insight into the mechanisms of communication in both domains. We expect to further conduct the work as described in the description of action until the end of the project, increasingly focusing on addressing objectives 2 and 3 over time.