Periodic Reporting for period 2 - DeepStore (The Dynamic Representational Nature of Working Memory)
Okres sprawozdawczy: 2023-07-01 do 2025-02-28
The project aims to understand how humans process information in Working Memory (WM). When information is temporarily maintained (or ‘stored’) in WM, the underlying brain processes do not resemble static storage, but are highly dynamic and constantly changing. The precise nature and functional role of these dynamics remains poorly understood. A critical factor in understanding the dynamics of WM processing is that the brain can represent any kind of memory contents (e.g. our car keys) at different levels of abstraction. In WM, our car keys might at any time take the form of a concrete mental image, an abstract conceptual idea, or a verbal label -- and recruit distinct brain areas and neural processes accordingly. Our project aims to understand how the brain juggles these multiple layers of endogenous representation in ways that enable powerful cognition and adaptive behaviour, despite WM's strictly limited storage capacity.
With its goal of delineating endogenous dynamics of WM information, the scope of the project is not limited to explicitly mnemonic functions but extends to cognitive information processing more generally (e.g. in perception, decision making, and transfer learning). Beyond its deliverables for the basic sciences, the project is expected to offer new insights into the role(s) of WM in learning and cognitive aging, and to provide new leverage points for the educational and clinical sectors in understanding and treating WM-associated deficiencies (e.g. in ADHD, dementia, depression, Parkinson’s, Alzheimer’s, and schizophrenic diseases). There is also public interest in understanding how memory works, including its frequent failures, and potential avenues for training and improvement. Another beneficiary can be AI research and its aspirations for better resource efficiency, where capacity-limited human WM can serve as a biological model system.
The main objectives of the project are (i) to uncover the spatiotemporal dynamics of abstraction and concretization in WM. The research program supports this goal at multiple levels of investigation, combining functional imaging, EEG/MEG, eye-tracking, behavioral models, and invasive recordings with novel representational geometry analysis approaches; (ii) to examine how WM-representational dynamics interact with long-term storage and learning. And (iii) to uncover the neural mechanisms underlying flexible representational transformations. Covering a wide range of paradigmatic WM settings, including single- and multi-item maintenance, (in-)attention and distraction, and varying task rules, the overarching goal is to deepen our understanding of how capacity-limited WM can supply the mind with just the right information, at just the right time, and in just the right format.
With its goal of delineating endogenous dynamics of WM information, the scope of the project is not limited to explicitly mnemonic functions but extends to cognitive information processing more generally (e.g. in perception, decision making, and transfer learning). Beyond its deliverables for the basic sciences, the project is expected to offer new insights into the role(s) of WM in learning and cognitive aging, and to provide new leverage points for the educational and clinical sectors in understanding and treating WM-associated deficiencies (e.g. in ADHD, dementia, depression, Parkinson’s, Alzheimer’s, and schizophrenic diseases). There is also public interest in understanding how memory works, including its frequent failures, and potential avenues for training and improvement. Another beneficiary can be AI research and its aspirations for better resource efficiency, where capacity-limited human WM can serve as a biological model system.
The main objectives of the project are (i) to uncover the spatiotemporal dynamics of abstraction and concretization in WM. The research program supports this goal at multiple levels of investigation, combining functional imaging, EEG/MEG, eye-tracking, behavioral models, and invasive recordings with novel representational geometry analysis approaches; (ii) to examine how WM-representational dynamics interact with long-term storage and learning. And (iii) to uncover the neural mechanisms underlying flexible representational transformations. Covering a wide range of paradigmatic WM settings, including single- and multi-item maintenance, (in-)attention and distraction, and varying task rules, the overarching goal is to deepen our understanding of how capacity-limited WM can supply the mind with just the right information, at just the right time, and in just the right format.
After establishing the ERC-funded research group, in the wake of Covid-19 restrictions, we started with carefully piloting the novel experimental paradigms and analysis framework of the research program. During piloting of experiments for our first work package, we discovered that under certain conditions, our visuospatial stimulus materials (object orientations) were associated with systematic pattens of miniature eye movements (microsaccades) which may pose a source of artefacts in neural recordings. This led us to modify and optimize our experimental setup to control for such artefacts in subsequent experiments. In parallel, we followed up on our incidental discovery in ocular activity by performing dedicated eye-tracking studies and extending our representational geometry framework to encompass gaze patterns. The first results of this line of work already provided preliminary support for key hypotheses of the research program, specifically in terms of dynamic changes in the level of WM abstraction upon encoding and retrospective cueing of visual memoranda. We published these first key findings in the multidisciplinary journal Nature Human Behaviour (Linde-Domingo and Spitzer, 2024).
Having optimized our base paradigm for neural recordings, we continued to collect EEG and fMRI data for several of the experiments in our first work package, which are currently under analysis and/or in preparation for publication. Interestingly, the preliminary results include novel evidence for generalized (“abstract”) WM representations in early visual cortex (V1; Yizhar et al., in prep). These findings join recent evidence from other labs (Kwak & Curtis, 2002) that WM may recruit early sensory areas even for representing high-level transformations (or abstractions) of the task-relevant stimulus information. In parallel work, using EEG and eye-tracking (Zorbek et al., in prep), we seek to characterize the precise time course of such reformatting and how it maps on the format(s) found in visual, parietal, and frontal areas, respectively.
Another important interim result from the first project period has been that active retrieval and/or reproduction (‘concretization’) of WM information can benefit its memorability and subsequent recall in long-term memory (Born and Spitzer, in prep.). Specifically, we found that long-term benefits of WM retrieval were particularly pronounced when the WM information was temporarily unattended, that is, when it had been retrieved from an deprioritized WM state. In ongoing and future work, we follow up on these findings to pinpoint the underlying neural mechanisms using EEG and fMRI, with a particular focus on task factors that may promote a (re-)concretization of the WM contents, such as retrospective cueing and active retrieval/recall.
Having optimized our base paradigm for neural recordings, we continued to collect EEG and fMRI data for several of the experiments in our first work package, which are currently under analysis and/or in preparation for publication. Interestingly, the preliminary results include novel evidence for generalized (“abstract”) WM representations in early visual cortex (V1; Yizhar et al., in prep). These findings join recent evidence from other labs (Kwak & Curtis, 2002) that WM may recruit early sensory areas even for representing high-level transformations (or abstractions) of the task-relevant stimulus information. In parallel work, using EEG and eye-tracking (Zorbek et al., in prep), we seek to characterize the precise time course of such reformatting and how it maps on the format(s) found in visual, parietal, and frontal areas, respectively.
Another important interim result from the first project period has been that active retrieval and/or reproduction (‘concretization’) of WM information can benefit its memorability and subsequent recall in long-term memory (Born and Spitzer, in prep.). Specifically, we found that long-term benefits of WM retrieval were particularly pronounced when the WM information was temporarily unattended, that is, when it had been retrieved from an deprioritized WM state. In ongoing and future work, we follow up on these findings to pinpoint the underlying neural mechanisms using EEG and fMRI, with a particular focus on task factors that may promote a (re-)concretization of the WM contents, such as retrospective cueing and active retrieval/recall.
The project introduces novel paradigms and an advanced analysis framework for studying WM abstraction in behaviour, M/EEG, fMRI, and eye-tracking. Having demonstrated the principled applicability of our approach, we proceed with applying it in large-scale to neural data sets in order to characterize the dynamics of WM abstraction during (in)attention, distraction, and multi-item maintenance. In our second work package, we focus on brain structures and mechanisms that support the transfer of WM information to long-term storage, and on the extent to which representational WM dynamics are susceptible to change over the lifespan. Finally, in our third and last work package, we wish to integrate direct neural recordings from experimental animals with non-invasive data from humans performing the same tasks, to help bridge persistent disparities between findings across species, especially with respect to the putative role(s) of prefrontal processes.