Periodic Reporting for period 2 - Neuroception (Neural Pathways from Recognition to Perception)
Berichtszeitraum: 2023-05-01 bis 2024-04-30
Object recognition is essential to our interaction with the external world. Our brain is able to effortlessly identify objects even under the highly dynamic conditions of natural vision in which even a single object can lead to an infinite number of different images on our retinas. This is a remarkable achievement of our visual system. Nevertheless, how the brain creates invariant object representations remains elusive. In this project we aim to investigate the neural information processing in higher visual cortical areas during object recognition. Specifically, we seek to (1) explore how visual stimuli are represented in the activity of neurons across the visual cortical hierarchy and how these representations evolve to incorporate semantic information, (2) characterize the feature selectivity of single cells using deep-learning algorithms, and (3) identify how interactions between the areas shape the neural representations. Collectively, these objectives are a highly innovative attempt to disentangle the roles of different higher cortical areas in object recognition.
In this project, we combine novel large-scale electrophysiological recordings and advanced behavioral methods to investigate neural information processing in higher visual cortical areas during object recognition.
In the first objective, we seek to investigate how cortical representations of visual stimuli progressively evolve across the cortical hierarchy to incorporate semantic information. For this, we have to record the neural activity of visual areas while mice are engaged in a visual discrimination task. Thus, we trained mice to discriminate between two different objects while they are freely-moving in their home cage. We are currently building a behavioral setup to enable us to record neural activity while mice are performing the task. In addition, we have a) established the recording technique using state-of-the-art high-density silicon probes that allow us to record the activity of hundreds of neurons simultaneously across many visual areas and b) optimized targeting and anatomical identification of the areas of interest.
Second objective is to characterize the selectivity of the neuronal populations in each cortical area. For this purpose, we utilize a closed loop approach that combines electrophysiological recordings with deep-learning modeling. Specifically, we simultaneously recorded the activity of hundreds of neurons in vivo in mouse visual areas in response to natural images. We then trained a convolutional neural network to predict the responses of each neuron recorded across the different areas and generated a set of stimuli that would optimally excite the recorded neurons. Subsequently, we showed the optimized stimuli back to the mice and recorded the activity of the same neurons. This enabled us to verify in vivo the results of the in silico model predictions. This approach allows us to identify what are the stimuli properties that the neural activity is invariant to across areas and thus to dissect the role of hierarchical processing in complex cortical computations such as object recognition.
Finally, the third objective focuses on resolving how dynamic interactions between areas shape the neural representations of visual stimuli. To identify the interareal interactions, we analyze the temporal dynamics of neural representations and compute the correlations of the neural activity between areas. We utilize advanced statistical modelling to uncover how the directionality of the neural information flow between the cortical areas contributes to shaping the object representations.
In the first objective, we seek to investigate how cortical representations of visual stimuli progressively evolve across the cortical hierarchy to incorporate semantic information. For this, we have to record the neural activity of visual areas while mice are engaged in a visual discrimination task. Thus, we trained mice to discriminate between two different objects while they are freely-moving in their home cage. We are currently building a behavioral setup to enable us to record neural activity while mice are performing the task. In addition, we have a) established the recording technique using state-of-the-art high-density silicon probes that allow us to record the activity of hundreds of neurons simultaneously across many visual areas and b) optimized targeting and anatomical identification of the areas of interest.
Second objective is to characterize the selectivity of the neuronal populations in each cortical area. For this purpose, we utilize a closed loop approach that combines electrophysiological recordings with deep-learning modeling. Specifically, we simultaneously recorded the activity of hundreds of neurons in vivo in mouse visual areas in response to natural images. We then trained a convolutional neural network to predict the responses of each neuron recorded across the different areas and generated a set of stimuli that would optimally excite the recorded neurons. Subsequently, we showed the optimized stimuli back to the mice and recorded the activity of the same neurons. This enabled us to verify in vivo the results of the in silico model predictions. This approach allows us to identify what are the stimuli properties that the neural activity is invariant to across areas and thus to dissect the role of hierarchical processing in complex cortical computations such as object recognition.
Finally, the third objective focuses on resolving how dynamic interactions between areas shape the neural representations of visual stimuli. To identify the interareal interactions, we analyze the temporal dynamics of neural representations and compute the correlations of the neural activity between areas. We utilize advanced statistical modelling to uncover how the directionality of the neural information flow between the cortical areas contributes to shaping the object representations.
This project aims to provide insights into the mechanisms underlying object perception by combining advanced behavioral training, large-scale electrophysiological recordings, and deep learning based computational modeling techniques. This multidisciplinary approach is necessary to build a holistic understanding of vision during ethologically relevant behaviors. Importantly, the proposed research will set a significant step towards understanding how different brain areas work concurrently to solve perceptual inference. As such, and based on the results of this project, future studies using natural stimuli will substantially advance our knowledge of how our brain is able to effortlessly identify objects even under the highly dynamic conditions during natural vision. Additionally, knowledge acquired by this project will ultimately advance artificial vision with social and technological impact. For instance, assistive devices for blind people will significantly benefit from invariant object recognition as it eliminates the prerequisite for correct alignment of the camera with objects. Equipping artificial vision with invariant object recognition is also critical to the advancement and safe use of other technologies that require high-fidelity vision, like self-driving cars.