Periodic Reporting for period 4 - CONTEXTVISION (Visual perception in Context)
Periodo di rendicontazione: 2020-10-01 al 2021-03-31
Everything occurs in a context. We see a car in the context of a street scene and a stove in the context of a kitchen. Context greatly helps the processing of individual objects. Surprisingly however, context hardly plays a role in most models of visual perception, which treat perception as a largely bottom-up categorization process.
In this research proposal, we examined how context changes the cortical computations that give rise to visual perception, focusing on contextual modulations in space and time. Moreover, we aimed to translate this research to a clinical condition that is marked by aberrant context modulations in perception: Autism Spectrum Disorder (ASD). This integrative approach has the potential to significantly advance theoretical models of perception, based on underlying neurobiology, and underline the importance of context for understanding perception. Moreover, the knowledge gleaned can have significant clinical impact, helping us to better understand the pathophysiology of ASD.
We have found that both spatial and temporal context have tremendous impact on sensory processing of current input. Spatial context leads to automatic filling-in and spreading of selection signals in early visual cortex. Similarly, temporal context leads to a sharper and more efficient neural representation of expected input in early visual cortex. These neural processes have strong consequences for perception and perceptual decision-making, leading to sensitive and stable perception of the visual world. Interestingly, we did not find any robust evidence for alterations of these contextual effects in ASD. Thereby, our results do not corroborate the influential hypo-prior theory of ASD.
In this research proposal, we examined how context changes the cortical computations that give rise to visual perception, focusing on contextual modulations in space and time. Moreover, we aimed to translate this research to a clinical condition that is marked by aberrant context modulations in perception: Autism Spectrum Disorder (ASD). This integrative approach has the potential to significantly advance theoretical models of perception, based on underlying neurobiology, and underline the importance of context for understanding perception. Moreover, the knowledge gleaned can have significant clinical impact, helping us to better understand the pathophysiology of ASD.
We have found that both spatial and temporal context have tremendous impact on sensory processing of current input. Spatial context leads to automatic filling-in and spreading of selection signals in early visual cortex. Similarly, temporal context leads to a sharper and more efficient neural representation of expected input in early visual cortex. These neural processes have strong consequences for perception and perceptual decision-making, leading to sensitive and stable perception of the visual world. Interestingly, we did not find any robust evidence for alterations of these contextual effects in ASD. Thereby, our results do not corroborate the influential hypo-prior theory of ASD.
Team member Ekman has shown temporally precise and spatially specific prediction signals in the primary visual cortex (Ekman et al, Nat Comm 2017). The pre-stimulus nature of these prediction signals was verified in a separate study (Kok, Mostert, de Lange PNAS 2017). These predictions appear to attenuate neural activity for incoming information (Richter, Ekman, de Lange J Neurosci 2018). Spatial object context serves as a prior that fills in corresponding retinotopic spatial regions (Ekman, Roelfsema, de Lange J Neurosci 2020), providing a neural implementation for spatial context effects.
Team member Fritsche has elucidated temporal context effects (Fritsche, Mostert, de Lange Curr Biol 2017), which can be opposite at different stages of the perceptual decision-making process. These effects are dependent on attention (Fritsche & de Lange JOV 2019), and they can be computationally understood as originating from an efficient encoding and Bayesian decoding strategy (Fritsche, Spaak, de Lange eLife 2020). We found opposite effects of choice history and evidence history, resolving a paradox of sequential choice bias (Bosch et al JOV 2020).
Finally, we found only limited support for the hypothesis stipulated in WP3 that predictions have distinct sensory consequences in individuals with ASD (Utzerath, Schmits, Buitelaar, de Lange Cortex 2018). Team member Bosch has examined similar prediction effects in temporal context. She found robust temporal context effects in ASD, which were not different from healthy controls. These results are currently still being written up.
Alongside these core projects, team member Perez-Bellido has also assisted in two studies of multisensory contextual modulations of perception, culminating in two publications (Luettke et al 2016, Luettke, Perez-Bellido et al 2018).
Altogether, the publications jointly underscore the importance of temporal and spatial context in perception and cognition, and explicate the neural and computational mechanisms by which these factors shape sensory and cognitive processes. These core computations appear largely unaltered in high functioning ASD patients, thereby not supporting a recent and influential hypothesis of Autism Spectrum Disorder.
Team member Fritsche has elucidated temporal context effects (Fritsche, Mostert, de Lange Curr Biol 2017), which can be opposite at different stages of the perceptual decision-making process. These effects are dependent on attention (Fritsche & de Lange JOV 2019), and they can be computationally understood as originating from an efficient encoding and Bayesian decoding strategy (Fritsche, Spaak, de Lange eLife 2020). We found opposite effects of choice history and evidence history, resolving a paradox of sequential choice bias (Bosch et al JOV 2020).
Finally, we found only limited support for the hypothesis stipulated in WP3 that predictions have distinct sensory consequences in individuals with ASD (Utzerath, Schmits, Buitelaar, de Lange Cortex 2018). Team member Bosch has examined similar prediction effects in temporal context. She found robust temporal context effects in ASD, which were not different from healthy controls. These results are currently still being written up.
Alongside these core projects, team member Perez-Bellido has also assisted in two studies of multisensory contextual modulations of perception, culminating in two publications (Luettke et al 2016, Luettke, Perez-Bellido et al 2018).
Altogether, the publications jointly underscore the importance of temporal and spatial context in perception and cognition, and explicate the neural and computational mechanisms by which these factors shape sensory and cognitive processes. These core computations appear largely unaltered in high functioning ASD patients, thereby not supporting a recent and influential hypothesis of Autism Spectrum Disorder.
The fMRI study of time-compressed preplay of anticipated events in human primary visual cortex (Ekman et al, Nat Comm 2017) employed a highly novel and unconventional use of fMRI imaging, employing ultra-high temporal resolution (TR=88 ms), alongside high spatial resolution in a restricted part of the brain. Usually fMRI measurements use TRs between 1500-3000 ms. This allowed us to temporally and spatially track anticipatory activity. This methodological innovation has generated a lot of interest within the field, and is being followed up on by several labs.
The MEG study of pre-stimulus templates (Kok et al, PNAS 2017) used innovative forward encoding models. All data and code have been made freely available, and the methods are being re-used by other researchers in the field.
The research on temporal context also went beyond the planned boundaries, by taking advantage of openly available neurophysiological data (from the Allen Institute Brain Observatory), to provide a stronger mechanistic basis for temporal context effects in sensory processing. Moreover, this research WP created a state-of-the-art computational observer model that can make use of temporal context in two distinct ways: efficient encoding of input, and Bayesian decoding of input.
The MEG study of pre-stimulus templates (Kok et al, PNAS 2017) used innovative forward encoding models. All data and code have been made freely available, and the methods are being re-used by other researchers in the field.
The research on temporal context also went beyond the planned boundaries, by taking advantage of openly available neurophysiological data (from the Allen Institute Brain Observatory), to provide a stronger mechanistic basis for temporal context effects in sensory processing. Moreover, this research WP created a state-of-the-art computational observer model that can make use of temporal context in two distinct ways: efficient encoding of input, and Bayesian decoding of input.