Periodic Reporting for period 2 - PrAud (Mesoscopic computational imaging of the predictive listening human brain)
Reporting period: 2023-01-01 to 2024-06-30
In everyday life, we experience an ever-changing environment. To deal with these dynamic changes and adjust our behaviour accordingly, a key function of our brain is to predict future states of the world. Predictive processes play a role in all our senses. In the auditory domain, the ability to form predictions of what we will hear next is fundamental when dealing with the everyday soundscape, as it helps us in segregating different sources in complex auditory scenes, to deal with incomplete information, and to make sense of sounds in noisy scenarios. In addition, aberrant predictive processing is hypothesized to underlie phantom sound perception such as in tinnitus or auditory hallucinations.
To support predictive processes, our brains are thought to infer causes of sensory stimuli by building internal generative models. In the auditory system, these computations are supported by information travelling from the sensory periphery to the cortex - extracting complex information from the acoustic content of sounds - as well as by extensive feedback processing which are hypothesised t convey the current best prediction of the sensory input supported by our internal generative model. The mismatch between the predicted and actual input (i.e. the prediction error) is used to update the generative model and travels from lower processing stages to higher brain areas.
So far, limitations in coverage and spatial resolution of non-invasive imaging methods have prohibited grounding contextual sound processing onto the fundamental computational units of the human brain, resulting in an incomplete understanding of its biological underpinnings. PrAud proposes to use state of the art technology - such as Ultra-High field functional Magnetic Resonance Imaging (UHF-fMRI) - to investigate how small subcortical structures and layers of cortex are involved in predictive hearing and in combination with magnetoencephalography, derive a neurobiological model of contextual sound processing at high spatial and temporal resolution.
To support predictive processes, our brains are thought to infer causes of sensory stimuli by building internal generative models. In the auditory system, these computations are supported by information travelling from the sensory periphery to the cortex - extracting complex information from the acoustic content of sounds - as well as by extensive feedback processing which are hypothesised t convey the current best prediction of the sensory input supported by our internal generative model. The mismatch between the predicted and actual input (i.e. the prediction error) is used to update the generative model and travels from lower processing stages to higher brain areas.
So far, limitations in coverage and spatial resolution of non-invasive imaging methods have prohibited grounding contextual sound processing onto the fundamental computational units of the human brain, resulting in an incomplete understanding of its biological underpinnings. PrAud proposes to use state of the art technology - such as Ultra-High field functional Magnetic Resonance Imaging (UHF-fMRI) - to investigate how small subcortical structures and layers of cortex are involved in predictive hearing and in combination with magnetoencephalography, derive a neurobiological model of contextual sound processing at high spatial and temporal resolution.
The hiring of personnel has proceeded as planned, we hired 2 PhD students that will work on the project for almost its entire duration and a PostDoc that has worked on the project for the first two years. In the first half of the project period, the team has collected and analysed data for several of the planned experiments. In particular, we have investigated predictive processes associated to sound frequency processing (Aim1 - Experiment 1). This dataset has been fully analysed and has resulted in a first publication looking at the possibility to use a novel method for fMRI data denoising in the context of auditory laminar experiments. The neuroscientific output related to this project already combines development on the biophysical model (Aim3), which has been developed to a point that we could use it on this first dataset. The results indicate that predictive frequency processing is not tonotopic (contrary to expectations) and that the processing of inputs that deviate in frequency from built expectations happens in superficial cortical layers of several cortical regions bilaterally. Deep cortical layers also present a modulation in their response but we hypothesise this response to be related to the update of the global model being built - an hypothesis that we are planning to follow up using a so called local-global paradigm (Aim 1.3) which we are already piloting. A publication with the main results from Aim1.1 fMRI study is expected to be submitted within 2024.
As we did not find frequency specificity for the responses, we have decided to investigate higher cortical stages not by considering the interaction between multiple acoustic cues as planned (Aim 2.2) but by employing a paradigm that uses natural sounds (syllables) that are thought to engage primarily secondary auditory cortical areas. We have chosen this paradigm, as in the collaborators network we have found synergy to perform this same experiments in epilepsy patients using ECoG thus providing more spatially accurate information than the originally proposed combination with MEG. This dataset has been fully collected already and its analysis is well underway.
In parallel to the experiments of Aim1, data have been collected using both UHF-fMRI and MEG for a projected oriented at understanding the influence of uncertainty in predictive auditory processing (Aim 2.2). The dataset is now complete and analyses are well underway for both he fMRI and MEG data. Preliminary results indicate a cortical layer dependent contribution of neural adaptation and uncertainty driven expectations. We are currently planning to write two manuscripts to report the results of the fMRI and MEG experiments. This project has also already generated an interesting follow up. Considering relevant literature in the field and in collaboration with a visiting master student, we have initiated pupillometry recordings for the stimuli we have generated. This is particularly interesting as pupillometry represents an inexpensive way to track the involvement of norepinephrine, which is thought to play a key role predictive processing by providing a signal indicates when a model needs resetting. We expect another publication to be devoted to the pupillometry study.
Data have also been collected for an associative learning paradigm in which participants form an association between a sequence of pictures and a particular sound. This paradigm allows reducing the influence of spurious haemodynamic effects while investigating the predictive influence of visual information on auditory processing as planned in Aim 2.3. Through the collaborators network, intracranial laminar electrophysiology and ECoG data collected with the same paradigm are available reaching a fundamentally finer scale of investigation compared to the originally planned combination with MEG. This project is in the last phase of analysis and preliminary results have been presented at a conference in The Netherlands in 2023 (the meeting of the Dutch Society for Brain and Cognition). This project is highly linked to work we are conducting from the computational point of view in which we are investigating novel computational approaches that allow predicting items in a series of tokens. We expect two publications to follow from this work.
Finally, a post doctoral researcher has been hired for the first two years to work primarily on the advancepemnnt of methods that allow linking computational models to brain imaging data. We have worked on a fundamental statistical issue that related to the use of out-of-sample testing and the correction of biases introduced by this approach. A publication for this work has been submitted and is available as a preprint.
As we did not find frequency specificity for the responses, we have decided to investigate higher cortical stages not by considering the interaction between multiple acoustic cues as planned (Aim 2.2) but by employing a paradigm that uses natural sounds (syllables) that are thought to engage primarily secondary auditory cortical areas. We have chosen this paradigm, as in the collaborators network we have found synergy to perform this same experiments in epilepsy patients using ECoG thus providing more spatially accurate information than the originally proposed combination with MEG. This dataset has been fully collected already and its analysis is well underway.
In parallel to the experiments of Aim1, data have been collected using both UHF-fMRI and MEG for a projected oriented at understanding the influence of uncertainty in predictive auditory processing (Aim 2.2). The dataset is now complete and analyses are well underway for both he fMRI and MEG data. Preliminary results indicate a cortical layer dependent contribution of neural adaptation and uncertainty driven expectations. We are currently planning to write two manuscripts to report the results of the fMRI and MEG experiments. This project has also already generated an interesting follow up. Considering relevant literature in the field and in collaboration with a visiting master student, we have initiated pupillometry recordings for the stimuli we have generated. This is particularly interesting as pupillometry represents an inexpensive way to track the involvement of norepinephrine, which is thought to play a key role predictive processing by providing a signal indicates when a model needs resetting. We expect another publication to be devoted to the pupillometry study.
Data have also been collected for an associative learning paradigm in which participants form an association between a sequence of pictures and a particular sound. This paradigm allows reducing the influence of spurious haemodynamic effects while investigating the predictive influence of visual information on auditory processing as planned in Aim 2.3. Through the collaborators network, intracranial laminar electrophysiology and ECoG data collected with the same paradigm are available reaching a fundamentally finer scale of investigation compared to the originally planned combination with MEG. This project is in the last phase of analysis and preliminary results have been presented at a conference in The Netherlands in 2023 (the meeting of the Dutch Society for Brain and Cognition). This project is highly linked to work we are conducting from the computational point of view in which we are investigating novel computational approaches that allow predicting items in a series of tokens. We expect two publications to follow from this work.
Finally, a post doctoral researcher has been hired for the first two years to work primarily on the advancepemnnt of methods that allow linking computational models to brain imaging data. We have worked on a fundamental statistical issue that related to the use of out-of-sample testing and the correction of biases introduced by this approach. A publication for this work has been submitted and is available as a preprint.
We have provided advancements in several research directions. In the field of fMRI data analysis we have evaluated a novel method for fMRI data demonising (NORDIC) specifically for its use in laminar fMRI data of the auditory cortex. This is particularly relevant as auditory cortex poses unique challenges to fMRI which generally result in lower signal to noise ratio (SNR). In this work we have propose an approach to evaluate the effect of NORDIC on fMRI data and outline some issues related to its use in low SNR data such as the ones we collect in PrAud. Within the same project we have validated the use of a biophysical model of laminar fMRI in order to inver neural dynamics from measured fMRI responses, an interesting approach as it allows to reduce the influence pf vascular biases to the laminar fMRI signal.
Within the laminar fMRI projects conducted so far we have strived for developing a reproducible data analysis pipeline. While some steps (i.e. the segmentation of cortical gray matter) still require manual intervention - we have now created a flexible approach which we are going to make public through the code repository of the group. Similarly, we will provide all analyses codes that allow us to extract relevant measures to predictive processes (e.g. predictions, prediction errors, uncertainty) from the stochastic stimuli which we are using in Aim2.2.
Finally, the development of an approach that allows correcting for biases in the coefficient of determination measured out-of-sample, while primarily driven from relevant question within this project (i.e. the need to link models of predictive processing to fMRI/MEG responses) will have large inflictions for the neuroimaging community. The use of out-of-sample test is in fact relatively common as it is believed to not suffer from biases induced by e.g. model dimensionality. Our results and theoretical advancements indicate that these assumptions are not correct - and that biased have to be taken into account when for example comparing models to one another. Also this contribution will be made publicly available as soon as the manuscript is ready.
From now to the end of the project we expect to publish all ongoing projects and derivatives. This includes results for Aim1.1 (two papers - one methodological and one on the main neuroscientific question); a paper for data that has been collected in relation to Aim 1.2 (either in combination with the parallel ECoG data collected by our collaborators or in two separate contributions); several manuscripts are expected to stem from the data collected for Aim 2.2 (one fMRI, one MEG and possibly one pupillometry manuscript); one manuscript on the Aim 2.3 dataset.
In addition we are now starting piloting data acquisition for a local/global paradigm (Aim1.3) and we expect to develop further the use of stochastic sequences in a followup experiment possibly considering the interaction with task demands (Aim2.2).
Within the laminar fMRI projects conducted so far we have strived for developing a reproducible data analysis pipeline. While some steps (i.e. the segmentation of cortical gray matter) still require manual intervention - we have now created a flexible approach which we are going to make public through the code repository of the group. Similarly, we will provide all analyses codes that allow us to extract relevant measures to predictive processes (e.g. predictions, prediction errors, uncertainty) from the stochastic stimuli which we are using in Aim2.2.
Finally, the development of an approach that allows correcting for biases in the coefficient of determination measured out-of-sample, while primarily driven from relevant question within this project (i.e. the need to link models of predictive processing to fMRI/MEG responses) will have large inflictions for the neuroimaging community. The use of out-of-sample test is in fact relatively common as it is believed to not suffer from biases induced by e.g. model dimensionality. Our results and theoretical advancements indicate that these assumptions are not correct - and that biased have to be taken into account when for example comparing models to one another. Also this contribution will be made publicly available as soon as the manuscript is ready.
From now to the end of the project we expect to publish all ongoing projects and derivatives. This includes results for Aim1.1 (two papers - one methodological and one on the main neuroscientific question); a paper for data that has been collected in relation to Aim 1.2 (either in combination with the parallel ECoG data collected by our collaborators or in two separate contributions); several manuscripts are expected to stem from the data collected for Aim 2.2 (one fMRI, one MEG and possibly one pupillometry manuscript); one manuscript on the Aim 2.3 dataset.
In addition we are now starting piloting data acquisition for a local/global paradigm (Aim1.3) and we expect to develop further the use of stochastic sequences in a followup experiment possibly considering the interaction with task demands (Aim2.2).