Periodic Reporting for period 4 - START (Spatio-Temporal Attention and Representation Tracking: the precise neural architecture of conscious object perception)
Période du rapport: 2022-07-01 au 2022-12-31
START aims to improve our understanding of consciousness by revealing and explaining the neural underpinnings of conscious access, capitalising on the combined strengths of multiple neuroimaging and behavioural measurement modalities. Using Representational Similarity Analyses (RSA) to combine brain data collected from Electroencephalography (EEG; precise in time), functional Magnetic Resonance Imaging (fMRI; precise in space), and behavioural tasks modulating conscious access and memory, START will fill an important gap in our understanding of the neural mechanisms involved in object recognition, from perception to conscious access and storage in long-term memory. Overall, the research on conscious access has three important challenges that START will aim to address:
(1) make use of novel neuroimaging tools to deepen and broaden insights into the spatiotemporal dynamics of conscious access.
(2) Develop innovative experimental designs to grasp knowledge on the mechanisms of amplification and encoding in working memory which are not yet fully understood.
(3) Consider the nature of internal brain representations in understanding the variability in task performance across trials and individuals.
Understanding the spatiotemporal mechanisms of object recognition and conscious access has important implications for our understanding of brain function, and potential translational applications towards better understanding disorders of consciousness.
(1) make use of novel neuroimaging tools to deepen and broaden insights into the spatiotemporal dynamics of conscious access.
(2) Develop innovative experimental designs to grasp knowledge on the mechanisms of amplification and encoding in working memory which are not yet fully understood.
(3) Consider the nature of internal brain representations in understanding the variability in task performance across trials and individuals.
Understanding the spatiotemporal mechanisms of object recognition and conscious access has important implications for our understanding of brain function, and potential translational applications towards better understanding disorders of consciousness.
WP1: The functional architecture of conscious access in time and space:
In this work package, we investigated how object representations in time and space affect conscious perception. Two main projects have been completed within this work package. These two projects have given us great insight into the role of visual features and neural representations (in time and space) of objects in conscious access. The manuscript prepared describing the first project has been published in Nature Communications, and the second project has been presented at the Visual Science Society conference 2019 and at Computational Cognitive Neuroscience 2019 and a manuscript is under preparation for peer-reviewed publlication.
WP2: Decoding target-related population code amplification.
This work package aimed at delineating how different attributes of visual stimuli are encoded and temporarily stored in the brain. Additionally, we were interested in investigating how task-relevance affects these representations. To this purpose, we combined the signals obtained through distinct EEG and fMRI recording sessions. This methodology allowed us to overcome the limitations of both techniques and achieves great spatial and temporal resolution. Results from this work package show how task influences the spatio-temporal processing of visual information in the brain, from low-level properties, to items being stored in working memory for later retrieval. A manuscript describing these results is being prepared.
WP3: inter-trial and inter-individual variability in conscious access:
This work package aimed at better understanding the inter-trial and inter-individual variability consistently observed in studies of conscious access. The data collection for this work package was planned when the COVID19 pandemic emerged. As a fallback, we collaborated with researchers from the University of Minnesota to collect the Natural Scenes Dataset, the largest 7 Tesla functional magnetic resonance imaging dataset to date, which aimed at investigating how the human brain processes visual natural scenes. The dataset is publicly available at naturalscenesdataset.org and a manuscript describing the dataset was published in 2022 (Allen et al. (2022). Nature Neuroscience). The scale of this dataset enabled developing novel computational models that describe brain activity in the visual brain from early level representations, to high level, semantic representations. A preprint for this manuscript is avaliable online (Doerig et al. (2022). ArXiv).
In this work package, we investigated how object representations in time and space affect conscious perception. Two main projects have been completed within this work package. These two projects have given us great insight into the role of visual features and neural representations (in time and space) of objects in conscious access. The manuscript prepared describing the first project has been published in Nature Communications, and the second project has been presented at the Visual Science Society conference 2019 and at Computational Cognitive Neuroscience 2019 and a manuscript is under preparation for peer-reviewed publlication.
WP2: Decoding target-related population code amplification.
This work package aimed at delineating how different attributes of visual stimuli are encoded and temporarily stored in the brain. Additionally, we were interested in investigating how task-relevance affects these representations. To this purpose, we combined the signals obtained through distinct EEG and fMRI recording sessions. This methodology allowed us to overcome the limitations of both techniques and achieves great spatial and temporal resolution. Results from this work package show how task influences the spatio-temporal processing of visual information in the brain, from low-level properties, to items being stored in working memory for later retrieval. A manuscript describing these results is being prepared.
WP3: inter-trial and inter-individual variability in conscious access:
This work package aimed at better understanding the inter-trial and inter-individual variability consistently observed in studies of conscious access. The data collection for this work package was planned when the COVID19 pandemic emerged. As a fallback, we collaborated with researchers from the University of Minnesota to collect the Natural Scenes Dataset, the largest 7 Tesla functional magnetic resonance imaging dataset to date, which aimed at investigating how the human brain processes visual natural scenes. The dataset is publicly available at naturalscenesdataset.org and a manuscript describing the dataset was published in 2022 (Allen et al. (2022). Nature Neuroscience). The scale of this dataset enabled developing novel computational models that describe brain activity in the visual brain from early level representations, to high level, semantic representations. A preprint for this manuscript is avaliable online (Doerig et al. (2022). ArXiv).
We developed and customised an array of cutting-edge software tools to manage, analyse, and disseminate the behavioural, EEG and fMRI data acquired throughout the START project. These include: automated structuring and metadata annotation of the data, scripted access to university high-performance computing (HPC) resources for faster data analysis, deployment of community-standardised analysis software, development of an advanced technique for increasing signal-to-noise ratio (SNR; Prince et al. (2022) Elife), and extended python-based representational similarity code (https://github.com/rsagroup/rsatoolbox(s’ouvre dans une nouvelle fenêtre)). Using modern software development and data scientific practices, these tools have greatly increased efficiency and reliability of parts of the data processing pipeline that previously were labour intensive and prone to human error.