Skip to main content

The spatio-temporal representational architecture of memory

Periodic Reporting for period 3 - STREAM (The spatio-temporal representational architecture of memory)

Reporting period: 2020-03-01 to 2020-07-31

Understanding human memory is one of the big challenges in neuroscience. People tend to believe that a good memory is one that preserves a maximally precise record or “snapshot” of the past. STREAM opposes this “snapshot” view, and is instead motivated by the premise that memory is a highly adaptive and reconstructive process. The major scientific goal of this project is to understand how memories of past events are reconstructed in the human brain. We are particularly interested in how the mnemonic reconstruction of an event systematically differs from the event that we originally experienced, and how this reconstruction process changes over time and when we repeatedly remember the same event. The project addresses these questions using novel behavioural tasks together with cutting-edge brain imaging (“decoding”) techniques. These methods allow us to decompose memories into their constituent elements (e.g. sensory details or the semantic gist of an event), and to observe where in the brain and when in time these elements are reactivated when participants are cued to remember complex episodes. The experimental work in STREAM rests on a novel framework of memory recall that we call the “reverse reconstruction hypothesis”. The central working hypothesis predicts that the human brain tends to reconstruct memories in a backwards fashion, such that the information flow is reversed compared to the original sensory experience (i.e. the encoding) of an event. Sensory information is known to be primarily processed along a detailed-perceptual to abstract-conceptual gradient when it first enters the brain. The reverse reconstruction hypothesis predicts that when information is retrieved from episodic memory, the various elements contained in a memory are reconstructed along a reversed gradient that prioritizes abstract-conceptual over detailed-perceptual information. Understanding how memories are reconstructed in the human brain, step by step, will constitute a major leap in our fundamental knowledge about memory. If the reverse reconstruction hypothesis is supported by this work, it also has important implications for clinical work, for example for understanding the over-generalization of memories in post-traumatic stress disorders.
All the evidence we have so far unanimously supports the reverse reconstruction view. This evidence comes from electrophysiological recordings, functional magnetic resonance imaging (fMRI), and various behavioural measures of memory performance in healthy human participants, as well as from intracranial electrophysiological recordings in epileptic patients. We have so far conducted 5 full behavioural experiments (>200 participants), 2 EEG studies, one simultaneous EEG-fMRI study, and have recorded 14 patients using the basic version of the STREAM memory reconstruction task. A very promising and clear picture is emerging from the work in this project, and has until now been published in 2 empirical papers and 1 review, as well as 14 conference contributions. A first milestone paper from the STREAM project was recently published in Nature Communications (Linde-Domingo et al., 2019), and covered the findings from our first 3 experiments. Using a novel reaction time paradigm and sophisticated pattern classification of EEG data, we demonstrate that when participants recall objects from long-term memory, high-level abstract information can be classified more rapidly than lower-level visual detail, while the opposite gradient is observed when participants visually perceive the same objects (as expected given the vision literature). The reversed reconstruction pattern was robustly observed in behavioural reaction times as well as neural activity patterns, strongly suggesting that memory recall prioritizes abstract-conceptual over detailed-perceptual information. We have replicated and extended this basic finding several times in the meantime, and presented the results at various international conferences. The second empirical paper (Kerrén et al., 2018) again provides evidence for a reversal of the information flow between perception and memory, demonstrating that the mnemonic patterns that are reactivated during memory recall rhythmically wax and wane along a slow oscillation, and that the content of a memory can be decoded at the opposite phase of this oscillation, compared with decoding of current perceptual inputs. This fundamental finding suggests that the human brain uses slow oscillations to separate in time the incoming perceptual from internally reactivated mnemonic information, a prediction made from animal and computational models of memory. We discuss some of these core findings and assumptions of STREAM in a review paper that was recently accepted for publication in Trends in Cognitive Sciences (Staresina & Wimber, in press). The data from several other experiments are still in the analysis or write-up stage, and first results have been presented at international conferences.
The results obtained so far within STREAM have moved the field a major step beyond what was previously known about the spatio-temporal architecture of memory reconstruction. The most important advance in knowledge comes from our findings which, for the first time, decompose memories into their constituent elements, and pinpoint exactly when in time distinct elements of a memory are reactivated in the brain and become consciously accessible. In the coming years, we expect to take forward these basic findings in several important ways, in line with the major objectives of the project. First, we are analysing the data from simultaneous EEG-fMRI recordings, where the excellent spatial resolution of MRI will allow us to map the (presumably reverse) memory reconstruction stream onto distinct brain areas. Second, we have collected a considerable number of datasets from epileptic patients who are implanted with intracranial electrodes for presurgical monitoring, and these recordings will allow us to determine the earliest memory reactivation processes that occur in the hippocampus (the most crucial area involved in episodic memory), as well as other neocortical areas. Third, we have so far focused on the mnemonic reconstruction of visual objects in isolation, and will in our future experiments introduce contextual elements (e.g. background scenes) to investigate how spatial-contextual information about an event is reconstructed from memory. Fourth, we have so far mainly used verbal cues (i.e. reminder words) to trigger the reactivation of memories, and we are working on follow-up experiments that vary the nature of the reminder used to elicit a memory, in order to understand how the type of reminder changes the memory reconstruction stream in brain and behaviour. Fifth, we have already started a series of experiments that test the hypothesis that memories become “semanticized” (i.e. more gist-like) over time and specifically with repeated remembering. Sixth, we are currently working on ways to externally interfere with the memory reconstruction stream to provide causal evidence, beyond the correlative evidence we have so far. By the end of this project, we expect that STREAM will provide a comprehensive set of empirical data that shed light onto how complex, multi-layered events are brought back to mind during memory recall.