Periodic Reporting for period 1 - NUMEROUS (NeUronal MEchanisms foR consciOUS perception)
Reporting period: 2023-01-01 to 2025-06-30
Understanding the mechanisms for consciousness is arguably one of the most intriguing questions of modern neuroscience. Why are some visual stimuli consciously perceived, whereas others remain subliminal? What is the relation between conscious perception, attention and working memory? Recent work of the Roelfsema lab demonstrated that weak but simple stimuli can be reported once they elicit a minimal level of activity in the frontal cortex, which is related to the storage of the stimulus in working memory (van Vugt et al., Science 2018). For these simple stimuli, the visual cortex acts as a relay station that needs to pass the information to higher cortical areas. However, we found that the perception of more complex visual stimuli relies on a more sustained interaction between visual cortex and associative brain regions, related to scene segmentation and visual attention (Kirchberger et al., 2021, Sci Adv.). These recent results pave the way for a genuine understanding of mechanisms for consciousness, inspiring new paradigms that assess the awareness of stimuli while varying the demands on attention and working memory. NUMEROUS compares the neuronal fate of simple and complex stimuli that do and do not enter consciousness across most brain regions. We measure the activity of single neurons in human patients who are implanted with electrodes as part of their treatment of drug-resistant epilepsy. We will also use a new method that permits the recording from numerous single neurons within an entire hemisphere of a monkey and characterize neuronal activity in most regions of the cortex and subcortex. We use perturbation methods at numerous locations throughout the brain in combination with functional imaging in monkeys to investigate the brain regions in which activity does (or does not) readily lead to a reportable experience. The combination of experiments of NUMEROUS will provide unprecedented insight into how sensory stimuli give rise to conscious perception.
The work in NUMEROUS takes place in 3 interrelated work-packages.
WP1 delineates the brain regions involved in tasks that probe conscious awareness in monkeys (Storm et al., Neuron, 2024; Roelfsema, Neuron, 2023). We recently published a large-scale data set called the THINGS ventral stream spiking dataset (TVSD), in which we extensively sampled neuronal activity in response to >25,000 natural images (Papale et al., Neuron, 2025). We anticipate that other researchers will use the TVSD to answer questions about neuronal tuning, analyse the interactions within and between cortical regions, and to compare spiking activity in monkeys to human neuroimaging data. We have also created a neural network models for the processes that shift attention across visual stimuli, modelling brain activity related to shifts of attention (Mollard et al., PLoS Comput. Biol., 2024), intimately related to entry into consciousness.
WP2 investigates memory. Items that are attended within working memory are in access awareness and we compare these items to the representation of memory items that are accessory, i.e. are relevant for a later task. We addressed this comparison by measuring the influence of pronouns on conceptual representations (Dijksterhuis et al., Science, 2024). We recorded from neurons in the human hippocampus during reading and found that concept cells that were selective to a particular noun were later reactivated by pronouns that refer to the cells’ preferred noun. These results provided new insight in how concept cells contribute to the reactivation of concepts in working memory, in particular when they become part of access consciousness because they are cued by the pronoun.
WP3 examines how well subjects are able to report the electrical activation of neurons in the different regions of the brain. The goal is to map the minimal currents necessary for the perception of electrical stimulation throughout the brain and examine the brain regions that are also activated with fMRI in monkeys and how they depend on the ability to report. We have accomplished the technical hurdles for implanting the stimulation electrodes in many positions in the brain. We will use a variant of the electrodes that have been described in Orlemann et al. (2024).
WP1 delineates the brain regions involved in tasks that probe conscious awareness in monkeys (Storm et al., Neuron, 2024; Roelfsema, Neuron, 2023). We recently published a large-scale data set called the THINGS ventral stream spiking dataset (TVSD), in which we extensively sampled neuronal activity in response to >25,000 natural images (Papale et al., Neuron, 2025). We anticipate that other researchers will use the TVSD to answer questions about neuronal tuning, analyse the interactions within and between cortical regions, and to compare spiking activity in monkeys to human neuroimaging data. We have also created a neural network models for the processes that shift attention across visual stimuli, modelling brain activity related to shifts of attention (Mollard et al., PLoS Comput. Biol., 2024), intimately related to entry into consciousness.
WP2 investigates memory. Items that are attended within working memory are in access awareness and we compare these items to the representation of memory items that are accessory, i.e. are relevant for a later task. We addressed this comparison by measuring the influence of pronouns on conceptual representations (Dijksterhuis et al., Science, 2024). We recorded from neurons in the human hippocampus during reading and found that concept cells that were selective to a particular noun were later reactivated by pronouns that refer to the cells’ preferred noun. These results provided new insight in how concept cells contribute to the reactivation of concepts in working memory, in particular when they become part of access consciousness because they are cued by the pronoun.
WP3 examines how well subjects are able to report the electrical activation of neurons in the different regions of the brain. The goal is to map the minimal currents necessary for the perception of electrical stimulation throughout the brain and examine the brain regions that are also activated with fMRI in monkeys and how they depend on the ability to report. We have accomplished the technical hurdles for implanting the stimulation electrodes in many positions in the brain. We will use a variant of the electrodes that have been described in Orlemann et al. (2024).
Our demonstration that pronouns activate hippocampal neurons that code for the concepts that the pronouns refer to (Dijksterhuis et al., 2024, Science) revealed a link between the activity of neurons in the human hippocampus and the representation of concepts and their interrelations during reading. The activity profile suggests that hippocampal neurons predominantly represent concepts which come in the focus of attention as a central topic of the narrative. These attended concepts could then be used for the retrieval of additional associations, a process to which the hippocampus contributes. How brain networks implement memory and syntactic computations is a topic for future research, which can now be investigated.
Furthermore, we presented the TVSD, a high-channel-count dataset of spiking activity in response to tens of thousands of natural images that enables the testing of hypotheses and models on the functions of the ventral stream in monkeys. This new dataset complements existing neuroimaging datasets based on natural image processing with spiking data with sub-millisecond precision because the neurophysiological signals were sampled at 30 kHz. We improved the design of a high-channel count cortical implant, enabling high density recordings across several areas of the ventral stream. The microelectrodes were implanted chronically, allowing us to present a large set of images across several recording days.
Furthermore, we presented the TVSD, a high-channel-count dataset of spiking activity in response to tens of thousands of natural images that enables the testing of hypotheses and models on the functions of the ventral stream in monkeys. This new dataset complements existing neuroimaging datasets based on natural image processing with spiking data with sub-millisecond precision because the neurophysiological signals were sampled at 30 kHz. We improved the design of a high-channel count cortical implant, enabling high density recordings across several areas of the ventral stream. The microelectrodes were implanted chronically, allowing us to present a large set of images across several recording days.