Periodic Reporting for period 4 - HOMEOSTASIS (The mental economy: testing a dynamic trade-off between internal storage and external sampling)
Okres sprawozdawczy: 2025-03-01 do 2025-08-31
While interacting with the external world, the brain can only represent very little of this world in working memory (WM). WM is therefore generally referred to as a limited-capacity system. This limitation is not a problem in daily life, however, because the external world typically remains available and can be accessed relatively easily. The current dominant theory of WM does not explain how the brain balances between internal storage and external sampling, as this theory exclusively relates to situations in which the remembered information is no longer physically present. The HOMEOSTASIS project is motivated by the idea that WM should be studied in interaction with the world that is still within view.
HOMEOSTASIS will develop a new theoretical model of WM based on an internal mental economy: we hypothesize that WM maintains a perceptual homeostasis by dynamically trading the costs of accurate internal storage against external sampling of the external visual world. Whereas current research on WM has a strong focus on its maximum capacity, this capacity may hardly be used as observers prefer to minimize internal storage due to the effortful nature of WM storage.We rigorously test the model’s theoretical basis using novel experimental paradigms in which WM is studied in interaction with the physically present environment. To decode the current content of WM, we adopt state-of-the-art electroencephalographic decoding techniques. Finally, we investigate patients with restricted deficits to specific components of the model and use machine learning techniques to discover biometric signatures in eye movements. This new model of WM will open a new window to diagnose WM disorders and for understanding how we interact with computer-manipulated virtual environments in an increasingly computer-dominated world.
The world that we consciously experience is an internal representation of the world, rather than the physical world itself. Through the different stages of perception, observers construct a model of the physical world by internalizing certain aspects of the visual information that is presented to our retina. Although research on visual perception has made great leaps forward in understanding which aspects of the visual world are selected by the brain for further processing, the HOMEOSTASIS project will fundamentally advance vision research: what determines which visual information is maintained in internal memory after it has been selected? This research program has various potential future applications:
First, the project will result in clear hypotheses about which properties of the environment determine the amount of information that will be internalized (e.g. reliability and familiarity). As the amount of internalized information is an index of the current memory load, this knowledge will be of great assistance in designing efficient (work) environments in which memory load is minimized.
Second, our clinical work will result in the development of a novel screening tool for specific dysfunctions in VWM by revealing the oculomotor features (biomarkers) that best reflect deficits in visual memory. Given the rise of easy-to-use and low-cost eye trackers, this approach is promising and feasible. In the near future, the cameras in mobile devices, such as tablets and smartphones, will be used to record eye movements, making such data widely available.
HOMEOSTASIS will develop a new theoretical model of WM based on an internal mental economy: we hypothesize that WM maintains a perceptual homeostasis by dynamically trading the costs of accurate internal storage against external sampling of the external visual world. Whereas current research on WM has a strong focus on its maximum capacity, this capacity may hardly be used as observers prefer to minimize internal storage due to the effortful nature of WM storage.We rigorously test the model’s theoretical basis using novel experimental paradigms in which WM is studied in interaction with the physically present environment. To decode the current content of WM, we adopt state-of-the-art electroencephalographic decoding techniques. Finally, we investigate patients with restricted deficits to specific components of the model and use machine learning techniques to discover biometric signatures in eye movements. This new model of WM will open a new window to diagnose WM disorders and for understanding how we interact with computer-manipulated virtual environments in an increasingly computer-dominated world.
The world that we consciously experience is an internal representation of the world, rather than the physical world itself. Through the different stages of perception, observers construct a model of the physical world by internalizing certain aspects of the visual information that is presented to our retina. Although research on visual perception has made great leaps forward in understanding which aspects of the visual world are selected by the brain for further processing, the HOMEOSTASIS project will fundamentally advance vision research: what determines which visual information is maintained in internal memory after it has been selected? This research program has various potential future applications:
First, the project will result in clear hypotheses about which properties of the environment determine the amount of information that will be internalized (e.g. reliability and familiarity). As the amount of internalized information is an index of the current memory load, this knowledge will be of great assistance in designing efficient (work) environments in which memory load is minimized.
Second, our clinical work will result in the development of a novel screening tool for specific dysfunctions in VWM by revealing the oculomotor features (biomarkers) that best reflect deficits in visual memory. Given the rise of easy-to-use and low-cost eye trackers, this approach is promising and feasible. In the near future, the cameras in mobile devices, such as tablets and smartphones, will be used to record eye movements, making such data widely available.
Our aim was to understand how people decide whether to store information internally in visual working memory or to retrieve it again from the world. We approached this question by treating behaviour as the outcome of a dynamic optimisation process in which the brain continuously weighs the cognitive effort of maintaining information against the effort required to access it externally.
We developed a suite of behavioural paradigms that allowed us to manipulate the costs of storage and external access independently. Across copying tasks, gaze-contingent methods, sequential sampling paradigms and controlled manipulations of perceptual availability, we found a remarkably strong and persistent preference for resampling information from the environment. Even when an internal representation would have been faster or more accurate, observers often chose to look back at the stimulus. When the effort or uncertainty associated with external access increased, participants shifted toward greater reliance on internal storage. Conversely, when access was easy, they reduced internal storage even at the expense of performance. These findings revealed that minimising cognitive effort is a key principle guiding the balance between memory and sampling. They also showed that commonly used behavioural indicators of memory use capture partly distinct aspects of this trade-off.
By combining eye-tracking with EEG decoding, we were able to measure how much information participants held in working memory while they actively sampled from their environment. At the neural level, we decoded both the load and the content of visual working memory during ongoing sampling behaviour. These results demonstrated that internal storage remains active even when participants rely behaviourally on the external world. Internal and external sources of information therefore operate jointly.
We investigated how people adapt their reliance on memory to environments that vary in their stability, reliability or predictability. Only when external information became unreliable in a way that substantially increased the cost of access did people shift toward stronger encoding and retention. These findings support the central idea that people typically exploit whichever source of information offers the lowest combined cognitive and physical cost at any given moment.
Analyses of fixation patterns, saccadic planning and pupil dilation revealed behavioural signatures that predicted how deeply information had been encoded, suggesting practical applications in cognitive assessment and neuropsychology.
We developed a suite of behavioural paradigms that allowed us to manipulate the costs of storage and external access independently. Across copying tasks, gaze-contingent methods, sequential sampling paradigms and controlled manipulations of perceptual availability, we found a remarkably strong and persistent preference for resampling information from the environment. Even when an internal representation would have been faster or more accurate, observers often chose to look back at the stimulus. When the effort or uncertainty associated with external access increased, participants shifted toward greater reliance on internal storage. Conversely, when access was easy, they reduced internal storage even at the expense of performance. These findings revealed that minimising cognitive effort is a key principle guiding the balance between memory and sampling. They also showed that commonly used behavioural indicators of memory use capture partly distinct aspects of this trade-off.
By combining eye-tracking with EEG decoding, we were able to measure how much information participants held in working memory while they actively sampled from their environment. At the neural level, we decoded both the load and the content of visual working memory during ongoing sampling behaviour. These results demonstrated that internal storage remains active even when participants rely behaviourally on the external world. Internal and external sources of information therefore operate jointly.
We investigated how people adapt their reliance on memory to environments that vary in their stability, reliability or predictability. Only when external information became unreliable in a way that substantially increased the cost of access did people shift toward stronger encoding and retention. These findings support the central idea that people typically exploit whichever source of information offers the lowest combined cognitive and physical cost at any given moment.
Analyses of fixation patterns, saccadic planning and pupil dilation revealed behavioural signatures that predicted how deeply information had been encoded, suggesting practical applications in cognitive assessment and neuropsychology.
A key contribution was the discovery of a strong and persistent preference for external sampling even in situations where internal storage would be advantageous. At the same time, the project showed that this sampling bias is not rigid. When the cost or uncertainty associated with external access increased, participants shifted toward stronger internal encoding. Memory use is shaped not by a fixed capacity limit but by an ongoing cost–efficiency optimisation. The project combined eye-tracking, EEG decoding and controlled manipulations of perceptual availability to study memory use under conditions that allow natural sampling. The project also contributed a set of behavioural and physiological markers that reveal how deeply information is encoded. Together, these methodological innovations extend the empirical and conceptual toolkit available to researchers studying visual working memory. HOMEOSTASIS had delivered a cohesive set of findings that establish a new view of visual working memory as an adaptive, context-sensitive resource. We demonstrated that internal and external sources of information operate jointly and that their relative contributions depend on the expected effort, accessibility and reliability of the environment. This framework unifies behavioural, neural and eye-movement evidence in a way that had not been achieved previously.