Final Report Summary - MULTISENSORYBRAIN (The Multisensory Human Brain - Solving the debate on direct and indirect pathways)
Project context and objectives
The overall objective of the project was to elucidate at which processing stage the sensory systems interact, and whether this stage depends on the context: the type of information ('what': stimulus eccentricity) and the goal of integration ('why': task demand). Furthermore, we explored the role of temporal context (rhythmicity in one or multiple sensory systems) in perception and multisensory integration.
Work performed
Using functional magnetic resonance imaging (fMRI), we tested the following propositions: 1. faster feedforward and lateral pathways subserve speed functions, such as detecting peripheral stimuli. Multisensory integration effects in this context are predicted in peripheral fields of low-level sensory cortices. 2. slower feedback pathways underpin accuracy functions, such as object discrimination. Integration effects in this context are predicted in higher order association cortices and central/foveal fields of low-level sensory cortex. Furthermore, we tested the hypothesis of different processing modes for rhythmic (predictable) vs. randomly occurring events by simultaneous fMRI and EEG (electro-encephalography) and subsequently using MEG (magneto-encephalography) to test the forthcoming hypothesis that rhythmicity interacts with multisensory integration.
fMRI results show that interactions of task demands and stimulus eccentricity in low-level sensory and higher order association cortices are more complex than would be predicted by the simple dichotomy of peripheral/speed and foveal/accuracy functions. Our findings point instead to a much more flexible, context-dependent and individuated use of the available neural circuitry. Granger causality (connectivity) analyses indicate task-specific connectivity between the sensory cortex and the higher order cortex and subcortical regions. Combined EEG-fMRI analysis revealed that fMRI activity in distinct parts of the superior temporal gyrus (STG) correlated with EEG power in the predicted dominant frequencies for rhythmic (low frequencies) and vigilant (high frequencies) processing modes. Finally, behavioural and MEG results reveal that temporal rhythmicity and multisensory input add up in improving detection.
Main results
The results of the different experiments provide a working model for multisensory integration in different contexts. The fMRI study addressed stimulus eccentricity and task demand; the EEG-fMRI and behavioural/MEG studies addressed temporal context. Taken together, the results show that the neural mechanism used for multisensory integration is highly flexible and determined by the situation at hand: the behavioural goal, stimulus eccentricity and the temporal predictability of the input. Finally, individual skills and strategies also profoundly influenced circuit selection for integration.
Socio-economic impact of the project
Clinical implications: The different activation patterns for processing identical information with different goals and vice versa indicate a high degree of flexibility. This supports the feasibility of interventions to train certain networks after, for example, the loss of one sense. Furthermore, we found significant individual differences in peripheral performance related to integration networks. This finding is highly relevant for cross-modal compensatory plasticity research, i.e. research into substitution devices for blind and deaf people. Finally, the EEG-fMRI results may help with the design of intervention programmes, for example, for schizophrenia patients based on audio-visual rhythmic information.
General advancement of knowledge: The revealed profound context-dependence of circuit use has important general implications for systems neuroscience: it underscores the idea that the brain in general is extremely flexible in selecting the most appropriate neural circuitry for each moments specific situation. It also emphasises the importance of considering individual differences in cognitive strategy and bias in brain circuitry in studying large-scale brain operations, a finding of high importance for future brain imaging studies.
The overall objective of the project was to elucidate at which processing stage the sensory systems interact, and whether this stage depends on the context: the type of information ('what': stimulus eccentricity) and the goal of integration ('why': task demand). Furthermore, we explored the role of temporal context (rhythmicity in one or multiple sensory systems) in perception and multisensory integration.
Work performed
Using functional magnetic resonance imaging (fMRI), we tested the following propositions: 1. faster feedforward and lateral pathways subserve speed functions, such as detecting peripheral stimuli. Multisensory integration effects in this context are predicted in peripheral fields of low-level sensory cortices. 2. slower feedback pathways underpin accuracy functions, such as object discrimination. Integration effects in this context are predicted in higher order association cortices and central/foveal fields of low-level sensory cortex. Furthermore, we tested the hypothesis of different processing modes for rhythmic (predictable) vs. randomly occurring events by simultaneous fMRI and EEG (electro-encephalography) and subsequently using MEG (magneto-encephalography) to test the forthcoming hypothesis that rhythmicity interacts with multisensory integration.
fMRI results show that interactions of task demands and stimulus eccentricity in low-level sensory and higher order association cortices are more complex than would be predicted by the simple dichotomy of peripheral/speed and foveal/accuracy functions. Our findings point instead to a much more flexible, context-dependent and individuated use of the available neural circuitry. Granger causality (connectivity) analyses indicate task-specific connectivity between the sensory cortex and the higher order cortex and subcortical regions. Combined EEG-fMRI analysis revealed that fMRI activity in distinct parts of the superior temporal gyrus (STG) correlated with EEG power in the predicted dominant frequencies for rhythmic (low frequencies) and vigilant (high frequencies) processing modes. Finally, behavioural and MEG results reveal that temporal rhythmicity and multisensory input add up in improving detection.
Main results
The results of the different experiments provide a working model for multisensory integration in different contexts. The fMRI study addressed stimulus eccentricity and task demand; the EEG-fMRI and behavioural/MEG studies addressed temporal context. Taken together, the results show that the neural mechanism used for multisensory integration is highly flexible and determined by the situation at hand: the behavioural goal, stimulus eccentricity and the temporal predictability of the input. Finally, individual skills and strategies also profoundly influenced circuit selection for integration.
Socio-economic impact of the project
Clinical implications: The different activation patterns for processing identical information with different goals and vice versa indicate a high degree of flexibility. This supports the feasibility of interventions to train certain networks after, for example, the loss of one sense. Furthermore, we found significant individual differences in peripheral performance related to integration networks. This finding is highly relevant for cross-modal compensatory plasticity research, i.e. research into substitution devices for blind and deaf people. Finally, the EEG-fMRI results may help with the design of intervention programmes, for example, for schizophrenia patients based on audio-visual rhythmic information.
General advancement of knowledge: The revealed profound context-dependence of circuit use has important general implications for systems neuroscience: it underscores the idea that the brain in general is extremely flexible in selecting the most appropriate neural circuitry for each moments specific situation. It also emphasises the importance of considering individual differences in cognitive strategy and bias in brain circuitry in studying large-scale brain operations, a finding of high importance for future brain imaging studies.