Periodic Reporting for period 2 - SURPRISE (How does surprise influence cognition and behavior?)
Reporting period: 2023-02-01 to 2024-07-31
Humans are ‘prediction machines’. We are exquisitely good at predicting future events and anticipating other agents. We owe this remarkable ability to the internal predictive models that we construct of ourselves and our environment, based on regularities in previous input. The tendency to seek and predict patterns in the environment is central to many aspects of human cognition.
Our internal predictive models are constantly evaluated and updated based on new, surprising input. Surprise (i.e. the distance between one’s prior and current beliefs) appears an essential ingredient in various cognitive faculties such as perception, learning, motivation and action and it strongly drives brain activity in both sub-cortical and cortical networks underlying goal-directed behavior.
The overall aim of this research proposal is to elucidate how surprise is computed within sensory circuits and how it influences information seeking behavior. To achieve this, I will: 1) test the theoretical proposition that surprise signals emerge from the discrepancy between prediction signals and input signals that are represented in distinct layers of the neocortex, using ultra-high field neuroimaging in human volunteers; 2) investigate how sensory surprise signals may be communicated to downstream areas to update the brain’s attentional sampling policies; and 3) investigate the relationship between sensory surprise and the explicit drive for information that we call curiosity.
Project SURPRISE has the broad general goal of understanding the neural generation and cognitive and behavioral consequences of sensory surprise signals. This is an ambitious goal, as it requires an integration from laminar circuit models that calculate sensory surprise to systems neuroscience and computational models of behavior, to investigate how agents utilitize surprise to influence information sampling. Thereby, the proposal bridges several levels of analysis (from circuit models to behavior) and several cognitive domains (perception, attention, motivation and curiosity). While this breadth is challenging, I hope that it will create exciting crosslinks between fields of research that are usually not connected, by studying a common principle - surprise - that may connect different levels of description and fields of cognition.
By embarking on a multi-scale investigation, from laminar circuit mechanisms to computational models of cognition and behavior, I hope to elucidate the basic mechanisms of surprise in the brain and its fundamental role in cognition. This may have important implications for understanding perception, learning and motivated cognition.
Our internal predictive models are constantly evaluated and updated based on new, surprising input. Surprise (i.e. the distance between one’s prior and current beliefs) appears an essential ingredient in various cognitive faculties such as perception, learning, motivation and action and it strongly drives brain activity in both sub-cortical and cortical networks underlying goal-directed behavior.
The overall aim of this research proposal is to elucidate how surprise is computed within sensory circuits and how it influences information seeking behavior. To achieve this, I will: 1) test the theoretical proposition that surprise signals emerge from the discrepancy between prediction signals and input signals that are represented in distinct layers of the neocortex, using ultra-high field neuroimaging in human volunteers; 2) investigate how sensory surprise signals may be communicated to downstream areas to update the brain’s attentional sampling policies; and 3) investigate the relationship between sensory surprise and the explicit drive for information that we call curiosity.
Project SURPRISE has the broad general goal of understanding the neural generation and cognitive and behavioral consequences of sensory surprise signals. This is an ambitious goal, as it requires an integration from laminar circuit models that calculate sensory surprise to systems neuroscience and computational models of behavior, to investigate how agents utilitize surprise to influence information sampling. Thereby, the proposal bridges several levels of analysis (from circuit models to behavior) and several cognitive domains (perception, attention, motivation and curiosity). While this breadth is challenging, I hope that it will create exciting crosslinks between fields of research that are usually not connected, by studying a common principle - surprise - that may connect different levels of description and fields of cognition.
By embarking on a multi-scale investigation, from laminar circuit mechanisms to computational models of cognition and behavior, I hope to elucidate the basic mechanisms of surprise in the brain and its fundamental role in cognition. This may have important implications for understanding perception, learning and motivated cognition.
Below I will outline the main results that have been obtained per WorkPackage (WP) by the scientific staff that has been employed on the grant.
WP1: How is surprise computed in sensory circuits?
We tackled this question from a variety of angles. First, we discovered that sensory expectations spread throughout the cortical hierarchy as we update our beliefs (Ferrari et al 2022). Activity modulations appear first in frontal and parietal regions, followed by the ventral visual stream, up to early visual cortex. Complementing these results, we discovered that in both humans and mice violations of expectations are observed throughout the visual hierarchy, and reflect high-level (conceptual) predictions, even in low-level visual cortex. We are currently validating the experimental design for a high-resolution functional neuroimaging study. Finally, we are expanding these findings to more naturalistic dynamic stimuli, observing a hierarchy of predictions during dynamic visual stimulation within the visual system.
WP2: How does surprise influence information sampling?
We examined the role of surprise in information sampling, adding a distinction between two forms of surprise, namely visual (sensory) surprise and conceptual (semantic) surprise. We studied how each of these determine information sampling by inspecting gaze behavior. The results have sparked great enthusiasm at an international conference, and are currently being written up. Moreover, we investigated spatial biases of attentional sampling using rapid invisible frequency tagging, unfortunately with inconclusive results.Finally we examined neural modulations in information sampling induced by surprise in a more naturalistic auditory context, showing that neural surprise was best predicted by computational models that incorporated long-term statistical learning, yet employed short context windows for prediction.
WP3: What is the relationship between curiosity and surprise?
The relationship between curiosity and surprise will be the main focus of the second half of the ERC grant. Initial results show that implicit markers of curiosity are strongly driven by specific forms of stimulus surprisal and entropy, in line with the hypothesis put forward in the proposal.
WP1: How is surprise computed in sensory circuits?
We tackled this question from a variety of angles. First, we discovered that sensory expectations spread throughout the cortical hierarchy as we update our beliefs (Ferrari et al 2022). Activity modulations appear first in frontal and parietal regions, followed by the ventral visual stream, up to early visual cortex. Complementing these results, we discovered that in both humans and mice violations of expectations are observed throughout the visual hierarchy, and reflect high-level (conceptual) predictions, even in low-level visual cortex. We are currently validating the experimental design for a high-resolution functional neuroimaging study. Finally, we are expanding these findings to more naturalistic dynamic stimuli, observing a hierarchy of predictions during dynamic visual stimulation within the visual system.
WP2: How does surprise influence information sampling?
We examined the role of surprise in information sampling, adding a distinction between two forms of surprise, namely visual (sensory) surprise and conceptual (semantic) surprise. We studied how each of these determine information sampling by inspecting gaze behavior. The results have sparked great enthusiasm at an international conference, and are currently being written up. Moreover, we investigated spatial biases of attentional sampling using rapid invisible frequency tagging, unfortunately with inconclusive results.Finally we examined neural modulations in information sampling induced by surprise in a more naturalistic auditory context, showing that neural surprise was best predicted by computational models that incorporated long-term statistical learning, yet employed short context windows for prediction.
WP3: What is the relationship between curiosity and surprise?
The relationship between curiosity and surprise will be the main focus of the second half of the ERC grant. Initial results show that implicit markers of curiosity are strongly driven by specific forms of stimulus surprisal and entropy, in line with the hypothesis put forward in the proposal.
The project has been overall very successful, as exemplified by already several manuscripts that have been accepted in prestigious journals (e.g. eLife, Journal of Neuroscience, Nature Reviews Psychology), attracting a wide audience of researchers. Many more projects are currently being written up or are currently being reviewed (see Major Achievements). The project has gone beyond the state of the art in several dimensions. It is closing the gap between non-invasive human and invasive animal neuroscience, by comparing neural modulations of surprise in humans and mice (by analyzing an openly available dataset). It is also embracing recent developments in Artificial Intelligence to provide a more computationally explicit formulation of predictability and surprise. Expected results of the second period of the grant include an elucidation of the relationship between surprise and implicit/explicit measures of curiosity, as well as developmental trajectories of behavioral and neural markers of surprise. This would potentially allow to better understand this important but poorly understood exploration-oriented behavioral state.