Skip to main content

Temporal Structure of Perception and Neuronal Stimulus Processing

Periodic Reporting for period 1 - TeSP (Temporal Structure of Perception and Neuronal Stimulus Processing)

Reporting period: 2018-05-01 to 2020-04-30

Most neuroscientific studies investigate sensory perception from a retrospective perspective, analyzing passive brain reactions after isolated stimulus presentations. Although this approach allows for well controlled experimental settings and has heralded a plethora of important findings on the neural basis of perception, it ignores two important factors that have been shown to significantly influence perception and neural processing of sensory information.
First, recent studies have shown that sensory perception across different modalities is significantly influenced by the ongoing neural activity present before presentation of sensory information. In other words, the state of the brain prior to the presentation of a stimulus critically influences how this stimulus will be perceived. Second, in the natural environment, stimuli are almost never presented in isolation, but are embedded in a certain context that holds predictive information about what stimuli to expect in the future. Recent scientific evidence shows that the human brain can make use of this contextual information in order to anticipate which kind of stimulus will appear given a certain context and thus effectively adapt behavioral output.
Taken together, contemporary scientific evidence confirms what von Helmholtz already suggested in 1860, namely that perception is an active inference process in which the brain continuously generates a model of its environment and compares this model against perceived sensory information. The current challenge perceptual neuroscience faces is to devise experimental paradigms and analysis techniques to investigate specifically how the brain implements this active perceptual process, which mechanisms it uses to take into account prior brain states and sensory information, and what are the consequences if these mechanisms are impaired?

The conceptualization of perception as a passive and reactive process ignores important recent findings on the active and generative nature of perception, often subsumed under the label predictive processing. This approach has offered a valuable wholistic explanation of neural stimulus processing in particular and brain function in general, which has been successfully implanted in both basic and clinical neuroscience.
However, the specific implementations of this framework are still relying mostly on very artificial paradigms, which hinders the transfer to real-life situations and effective clinical applications. A systematic investigation of the active process of perception using paradigms that closely resemble conditions met in the natural environment promises a better understanding of how perception shapes our real-life experience of the world. In addition, research on this topic will provide insight in what happens when active perception is impaired, which has been shown to be the case in multiple neurological and psychiatric diseases.

The current project overcomes the above-mentioned limitations by studying how perception and neural processing of a target stimulus is influenced by previously presented stimuli that contain predictive information about the target stimulus. This provides an opportunity to investigate how the brain reads out and integrates past sensory information in order to predict future stimuli. Specifically, the current project investigates how the brain determines what past sensory information is taken into account to form predictions about future stimuli and how this is mechanistically implemented, thereby providing insights into the temporal structure of neural stimulus processing and predictive processing of future stimuli. Importantly, the paradigm employed in the current project presents predictive information in a way that closely mimics how predictive information is presented in the natural environment, thereby maximizing ecological validity and transfer to real-life applications.
To this end, the current project uses magnetoencephalography (MEG) recordings of human brain activity, which allow to analyze neural activity with high temporal precision. Further, the current project spatially localizes important components of these integration and prediction processes by means of electrocorticographic (ECoG) recordings.
Since the beginning of the outgoing phase hosted at the New York University Langone Medical Center, I have acquired and analyzed MEG recordings of twenty subjects working on a novel auditory prediction paradigm. In this task, subjects have to predict upcoming tone pitches after being presented with a tone sequence containing a particular pattern of tones. The main result is that the brain exhibits activity changes that are predictive of future tones, thereby providing a direct neural correlate of an active and generative process underlying sensory perception. Further, to generate such a prediction, the brain flexibly integrates past information over a specific amount of information, instead of integrating over a fixed period of time. The data acquisition and analysis are completed and the corresponding manuscript has been submitted to a peer-reviewed neuroscientific journal.
Next, I have implemented a similar task in six epileptic patients being implanted with ECoG recordings, which allows an unambiguous spatial localization of the underlying perceptual mechanisms. While the analysis pipeline is completed, I am currently waiting to further increase the number of subjects, since patient acquisition has been significantly impaired by complications related to the recent COVID19-outbreak.
In collaboration with my European beneficiary, I have completed and published two papers on the temporal resolution of sensory perception and the underlying neural mechanisms in patients with hepatic encephalopathy, a patient group that is known to exhibit perceptual alterations and changes in ongoing neural activity measures.
Both the work performed at my US partner organization and at my European beneficiary has been presented at the annual meeting of the society of neuroscience in form of an invited talk. I further presented my ECoG project at the highly competitive OIST Computational Neuroscience Course in Okinawa, Japan.
The present project substantially adds to the current state of the art in perceptual neuroscience by its use of a paradigm capturing predictive information common in natural settings and its unprecedented result on how the brain integrates past sensory input to predict future stimuli. Further expected results until the end of the project include an unambiguous localization and distinction of the neural mechanisms underlying the active predictive processing of sensory information by means of ECoG, which will represent promising targets for future basic and clinical research. In the intermediate run, the corresponding experiences will help the recruited researcher to apply for funding to establish his own research group, whose work will further extend the current findings and directly apply this line of research to psychiatric patient samples. On the long term, the present results will add to a better understanding of sensory perception as an active and generative process that heavily relies on prior brain states and contextual sensory information, while further aiding to implement this realization in the next generation of experimental paradigms used to study the neural basis of sensory perception.
Summary of findings from Study1