Periodic Reporting for period 4 - NEUROMEM (A Neurocomputational Model of Episodic Memory)
Reporting period: 2021-04-01 to 2022-09-30
Our memories define us, and their disruption in psychiatric and neurological conditions can be devastating. However, how we are able, e.g. to remember our wedding day and re-imagine the scene that was around us, remains one of the great mysteries of the human mind. NEUROMEM is an integrated experimental and computational attempt at a fundamental breakthrough in this problem. Building on recent insights into how environmental location and orientation is encoded by neurons in the mammalian brain, we aim to develop a mechanistic understanding of how events we experience are stored, recalled and imagined, i.e. a neurocomputational model of how specific memories result from patterns of activity in neuronal populations. We hope this will generate new hypotheses and explanations at the cognitive level, of interest to all scholars of the complexity of the human mind, and allow neurophysiological interpretation of behavioural data - providing a vital link between cognitive theory and neuroimaging and neurological data. Its implications extend beyond memory, including the mechanism for imagining views that have not been experienced.
We developed a detailed model of spatial memory and imagery, incorporating representations of objects into egocentric parietal and allocentric medial temporal representations to combine the content and context of an experience within flexible representations (Bicanski and Burgess, 2018). This new model offers an account on how the brain stores complex representations in memory, via associating the content of an experience with the surrounding context, and can flexibly use these representations to generate imagery to guide future behaviour. This model also incorporates the provision for imagined movement via connections between grid cells in entorhinal cortex and hippocampal place cells. The model is important in consolidating our understanding and making predictions about how memories are formed, retrieved and updated within a complex system of brain regions.
To investigate and model the interactions between place cells and grid cells to support dynamic imagery and memory representations, we have proposed a model of recognition memory in which grid cells encode translation vectors between features of an attended stimulus and thus guide eye movements between expected features to accumulate evidence to identify experienced stimuli (Bicanski and Burgess, 2019). We also modelled the interactions between place and grid cells for optimally inferring our environmental location (Evans & Burgess, NeurIPS, 2019).
We developed a rich setting (based on Harry Potter’s Hogwarts) in immersive and desk-top virtual reality (VR) to test memory for object-locations in different contexts, and combined this with fMRI to investigate its neural bases. We taught participants routes through a ”memory palace” within this environment to memorise long lists of items in order, and demonstrated a role for grid cells in this process by predicting fMRI activity in entorhinal cortex as a function of the orientation of each participant’s grid-like patterns during navigation relative to the orientation of the routes that they used (Constantinescu, Castegnaro et al., in prep). We also identified the brain regions supporting learning of the spatial context of a fearful stimulus (Suarez-Jimenez et al., 2018), and examined the effects of semantic and temporal similarity in defining the context of list elements and their differential neural bases (Convertino, Geerts & Burgess, in prep), and performed an fMRI experiment to test for grid-coding of semantic and temporal context (Convertino, Constantinescu & Burgess, in prep). We then developed a general model of context-dependent learning and memory in a wide variety of tasks, in collaboration with Sam Gershman and Kim Stachenfeld (Geerts, Gershman, Burgess, Stachenfeld, Psychological Review, in press).
We examined memory for multi-element events, and showed that both retrieval of presented associations and inference of missing ones reflect a process of pattern completion. This process was not affected by repeated presentations of individual associations and was consistent with an attractor network simulation of CA3 (Ihksan et al., 2020). We also showed that overlapping events undergo pattern separation in memory (Zotow & Burgess, 2020), that negative emotional content reduces the coherence of memory for the events (Bisby et al., 2018), and that a long delay it did not impair new items being associated to the previously encoded items and forming a closed loop of associations (Joensen et al., in prep).
We investigated the consolidation into long term memory of episodes (short video clips), showing that deliberate memory for the clips after one week was improved by brief wakeful rest after encoding, whereas intrusive thoughts them during the week were reduced (Horlyck et al., 2019). In this same paradigm, we found that hippocampal activity during the rest period correlates with subsequent deliberate memory whereas activity in the amygdala correlated with intrusive thoughts (Horlyck et al., in prep).
We have developed a model of the consolidation of episodic memories into semantic memory, including how memories are reconstructed or new are imagined, and explaining the nature of gist-based distortions and the effects of hippocampal damage (Spens & Burgess, submitted).
To investigate and model the interactions between place cells and grid cells to support dynamic imagery and memory representations, we have proposed a model of recognition memory in which grid cells encode translation vectors between features of an attended stimulus and thus guide eye movements between expected features to accumulate evidence to identify experienced stimuli (Bicanski and Burgess, 2019). We also modelled the interactions between place and grid cells for optimally inferring our environmental location (Evans & Burgess, NeurIPS, 2019).
We developed a rich setting (based on Harry Potter’s Hogwarts) in immersive and desk-top virtual reality (VR) to test memory for object-locations in different contexts, and combined this with fMRI to investigate its neural bases. We taught participants routes through a ”memory palace” within this environment to memorise long lists of items in order, and demonstrated a role for grid cells in this process by predicting fMRI activity in entorhinal cortex as a function of the orientation of each participant’s grid-like patterns during navigation relative to the orientation of the routes that they used (Constantinescu, Castegnaro et al., in prep). We also identified the brain regions supporting learning of the spatial context of a fearful stimulus (Suarez-Jimenez et al., 2018), and examined the effects of semantic and temporal similarity in defining the context of list elements and their differential neural bases (Convertino, Geerts & Burgess, in prep), and performed an fMRI experiment to test for grid-coding of semantic and temporal context (Convertino, Constantinescu & Burgess, in prep). We then developed a general model of context-dependent learning and memory in a wide variety of tasks, in collaboration with Sam Gershman and Kim Stachenfeld (Geerts, Gershman, Burgess, Stachenfeld, Psychological Review, in press).
We examined memory for multi-element events, and showed that both retrieval of presented associations and inference of missing ones reflect a process of pattern completion. This process was not affected by repeated presentations of individual associations and was consistent with an attractor network simulation of CA3 (Ihksan et al., 2020). We also showed that overlapping events undergo pattern separation in memory (Zotow & Burgess, 2020), that negative emotional content reduces the coherence of memory for the events (Bisby et al., 2018), and that a long delay it did not impair new items being associated to the previously encoded items and forming a closed loop of associations (Joensen et al., in prep).
We investigated the consolidation into long term memory of episodes (short video clips), showing that deliberate memory for the clips after one week was improved by brief wakeful rest after encoding, whereas intrusive thoughts them during the week were reduced (Horlyck et al., 2019). In this same paradigm, we found that hippocampal activity during the rest period correlates with subsequent deliberate memory whereas activity in the amygdala correlated with intrusive thoughts (Horlyck et al., in prep).
We have developed a model of the consolidation of episodic memories into semantic memory, including how memories are reconstructed or new are imagined, and explaining the nature of gist-based distortions and the effects of hippocampal damage (Spens & Burgess, submitted).
The model we developed (Bicanski & Burgess, 2018) provides a new, neuronal-level, understanding of visuospatial episodic memory and imagery, from the experience of visualizing a scene from a previously occupied viewpoint or imagining it from a new viewpoint, to interpreting fMRI activity or effects of lesions and the ability to plan new trajectories. This represents a breakthrough in terms of explaining complex cognitive, functional neuroimaging and neuropsychological data in terms of the underlying neuronal mechanisms.
Our review of vector coding provides a theoretical framework for several types of spatial coding neurons, including some cell types being discovered after their prediction in the 2018 model (Bicanski & Burgess, 2019).
Our experiment on multi-element events formed from overlapping associations (Bisby et al., 2018) showed that negative emotional content reduces the coherence of memory for the events. This has potentially important implications for the symptomatology of post-traumatic stress disorder (implying that traumatic memories are remembered differently to very strong neutral episodic memories).
The immersive virtual reality spatial memory task we developed was successfully adapted to provide a test for early detection of Alzheimer's disease, with extremely successful results (Howett*, Castegnaro* et al., Brain, 2019).
Our virtual spatial navigation paradigm was also combined with magneto-encephalography and applied to discover two interesting aspects of neural processing in Schizophrenia (again, a fortuitous application of the methodology we developed). We found that Schizophrenic patients lacked the theta-band coherence usually seen between medial temporal and medial prefrontal areas during spatial memory recall (Adams*, Bush* et al., Brain, 2020), and that they lacked the grid-like modulation of theta power usually seen during spatial navigation (Convertino*, Bush* et al., Brain, Brain, 2022).
Our review of vector coding provides a theoretical framework for several types of spatial coding neurons, including some cell types being discovered after their prediction in the 2018 model (Bicanski & Burgess, 2019).
Our experiment on multi-element events formed from overlapping associations (Bisby et al., 2018) showed that negative emotional content reduces the coherence of memory for the events. This has potentially important implications for the symptomatology of post-traumatic stress disorder (implying that traumatic memories are remembered differently to very strong neutral episodic memories).
The immersive virtual reality spatial memory task we developed was successfully adapted to provide a test for early detection of Alzheimer's disease, with extremely successful results (Howett*, Castegnaro* et al., Brain, 2019).
Our virtual spatial navigation paradigm was also combined with magneto-encephalography and applied to discover two interesting aspects of neural processing in Schizophrenia (again, a fortuitous application of the methodology we developed). We found that Schizophrenic patients lacked the theta-band coherence usually seen between medial temporal and medial prefrontal areas during spatial memory recall (Adams*, Bush* et al., Brain, 2020), and that they lacked the grid-like modulation of theta power usually seen during spatial navigation (Convertino*, Bush* et al., Brain, Brain, 2022).