Periodic Reporting for period 1 - SPEEDY (Defining an integrated model of the neural processing of speech in light of its multiscale dynamics)
Okres sprawozdawczy: 2022-10-01 do 2025-03-31
This interdisciplinary project defines an integrated model of speech processing by recording, modelling and manipulating neural oscillatory dynamics during perception of speech defined as a multiscale temporal signal. Dominant models of speech perception describe its underlying neural mechanisms at a static neuroanatomical level, neglecting the cognitive algorithmic and neural dynamic levels of description. These latter can only be investigated by considering the temporal dimension of speech, which is structured according to a hierarchy of linguistic timescales (phoneme, syllable, word, phrase...). Recent advances in behavioural paradigms, computational modelling, and neuroimaging data analysis make it now possible to investigate the cognitive algorithms and neural dynamics subtending the processing of speech. To define an integrated model of speech perception, we: 1. record neural activity in humans with magnetoencephalography (MEG) and intracranial electroencephalography (EEG) recordings during perception of continuous speech; 2. quantify linguistic information at each timescale of speech with a computational large language model (LMM, transformer); and 3. estimate their respective and shared neural correlates with multivariate and directed connectivity analyses. Feasibility is ensured by an in-house access to neuroimaging and intracranial recordings. This project critically tests whether neural oscillations play a fundamental role in the computational processes of perception and cognition. It defines the mapping between speech and neural timescales and address the question of how information is transferred and combined along the linguistic computational processing hierarchy. Overall, it specifies –in terms of the nature of the information processed and of the dynamical hierarchical organisation– the respective contributions of left and right hemispheric ventral and dorsal auditory pathways in speech processing.
Work Package 1: Hierarchy of neural computations dedicated to linguistic information processing.
Activities:
- Development of a novel compressed speech presentation paradigm to efficiently estimate comprehension thresholds for different linguistic features simultaneously [Giroud_2023_Cognition].
- Development of novel analytical techniques for intracranial EEG data to examine speech and music processing, including methods for quantifying shared vs. selective neural responses across frequency bands [teRietmolen_2024_eLife].
- Developing an approach to characterize the evolution of density maps obtained by dimensionality reduction over time to study the complexity of brain dynamics during speech and music listening [Runfola_2024_NetNeuro].
Main Achievements:
- Estimation of channel capacities associated with different linguistic features: acoustic modulation and syllabic rate around 15 Hz, and phonemic rate around 35 Hz. This highlights processing constraints at different levels of linguistic analysis [Giroud_2023_Cognition].
- Identification that contextual lexical information is the main cognitive feature limiting the flow of natural speech, as it is already close to its channel capacity at normal speech rates [Giroud_2023_Cognition].
Work Package 2: Dissociation of linguistic and acoustic temporal information.
Activities:
- Using neuroimaging recordings during natural speech perception to study the impact of phonemic and syllabic rates on sentence processing [Giroud_2024_SciAdv].
- Exploring the conditions under which motor areas optimally contribute to speech perception by manipulating the signal-to-noise ratio and speech rate in a behavioral paradigm [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Development of a computational model of the dorsal auditory pathway [Zalta_2024_SciAdv].
Main Achievements:
- Highlighting the role of motor activity in speech perception in noise, suggesting overlapping processes and shared neural resources between auditory and motor functions [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Proposing a computational model describing the computations of the dorsal auditory pathway, implicated in predictive timing during auditory perception [Zalta_2024_SciAdv].
- Revealing a mechanism by which the brain is capable of tracking the speech signal at the phonemic timescale, although its sufficient and necessary roles for comprehension have not been determined [Giroud_2024_SciAdv].
Work Package 3: Dissociation of linguistic and acoustic spectral information.
Activities:
- Generalizing the framework of auditory hemispheric asymmetry to environmental sounds by studying temporal and spectral modulations [Robert_2024_CerebralCortex].
- Investigating alpha oscillations in the auditory cortex and their dynamics in response to different auditory stimuli [Lopez_2024_CommsBiol].
- Exploring neural dynamics in naturalistic contexts of speech and music production and perception using intracranial EEG [Lorenz_2025_Cortex, Runfola_2024_NetNeuro].
Main Achievements:
- Demonstration that the processing of speech and music largely relies on shared neural resources and frequency-specific distributed networks rather than anatomically distinct regions [teRietmolen_2024_eLife].
- Proposing an ecological framework for the origin of auditory hemispheric asymmetry, not limited to speech and music, but also including environmental sounds [Robert_2024_CerebralCortex].
- Studying the characteristics of the power spectrum (peak frequency, relative power, and aperiodic component) in the auditory cortex in response to different types of sounds [Lopez_2024_CommsBiol].
- Observing that brain activity explores more states during speech listening than during music listening, and fewer during rest, suggesting that complex cognitive functions require more intricate brain dynamics [Runfola_2024_NetNeuro].
- Studying a rare case of a professional musician implanted with intracranial EEG, providing detailed insights into the neural processes underlying speech and music production and perception [Lorenz_2025_Cortex].
Work Package 4: Modulation of Speech Processing by Motor Activity
Activities:
- Investigate how overt motor tracking at different rates (syllabic, word, phrasal) influences speech comprehension under varying acoustic conditions.
Main achievements:
-Motor priming (tapping) influences speech comprehension, and performance in degraded speech is positively correlated with perception-production coupling abilities [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Rhythmic tapping synchronized to the lexical rate of speech significantly improves speech perception in noise [teRietmolen_2025_PRSB].
Conclusion:
These results demonstrate that the "SPEEDY" project has made significant advances in understanding the neural processing of speech and its multiscale dynamics, achieving several of the objectives outlined in the initial work packages.
Activities:
- Development of a novel compressed speech presentation paradigm to efficiently estimate comprehension thresholds for different linguistic features simultaneously [Giroud_2023_Cognition].
- Development of novel analytical techniques for intracranial EEG data to examine speech and music processing, including methods for quantifying shared vs. selective neural responses across frequency bands [teRietmolen_2024_eLife].
- Developing an approach to characterize the evolution of density maps obtained by dimensionality reduction over time to study the complexity of brain dynamics during speech and music listening [Runfola_2024_NetNeuro].
Main Achievements:
- Estimation of channel capacities associated with different linguistic features: acoustic modulation and syllabic rate around 15 Hz, and phonemic rate around 35 Hz. This highlights processing constraints at different levels of linguistic analysis [Giroud_2023_Cognition].
- Identification that contextual lexical information is the main cognitive feature limiting the flow of natural speech, as it is already close to its channel capacity at normal speech rates [Giroud_2023_Cognition].
Work Package 2: Dissociation of linguistic and acoustic temporal information.
Activities:
- Using neuroimaging recordings during natural speech perception to study the impact of phonemic and syllabic rates on sentence processing [Giroud_2024_SciAdv].
- Exploring the conditions under which motor areas optimally contribute to speech perception by manipulating the signal-to-noise ratio and speech rate in a behavioral paradigm [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Development of a computational model of the dorsal auditory pathway [Zalta_2024_SciAdv].
Main Achievements:
- Highlighting the role of motor activity in speech perception in noise, suggesting overlapping processes and shared neural resources between auditory and motor functions [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Proposing a computational model describing the computations of the dorsal auditory pathway, implicated in predictive timing during auditory perception [Zalta_2024_SciAdv].
- Revealing a mechanism by which the brain is capable of tracking the speech signal at the phonemic timescale, although its sufficient and necessary roles for comprehension have not been determined [Giroud_2024_SciAdv].
Work Package 3: Dissociation of linguistic and acoustic spectral information.
Activities:
- Generalizing the framework of auditory hemispheric asymmetry to environmental sounds by studying temporal and spectral modulations [Robert_2024_CerebralCortex].
- Investigating alpha oscillations in the auditory cortex and their dynamics in response to different auditory stimuli [Lopez_2024_CommsBiol].
- Exploring neural dynamics in naturalistic contexts of speech and music production and perception using intracranial EEG [Lorenz_2025_Cortex, Runfola_2024_NetNeuro].
Main Achievements:
- Demonstration that the processing of speech and music largely relies on shared neural resources and frequency-specific distributed networks rather than anatomically distinct regions [teRietmolen_2024_eLife].
- Proposing an ecological framework for the origin of auditory hemispheric asymmetry, not limited to speech and music, but also including environmental sounds [Robert_2024_CerebralCortex].
- Studying the characteristics of the power spectrum (peak frequency, relative power, and aperiodic component) in the auditory cortex in response to different types of sounds [Lopez_2024_CommsBiol].
- Observing that brain activity explores more states during speech listening than during music listening, and fewer during rest, suggesting that complex cognitive functions require more intricate brain dynamics [Runfola_2024_NetNeuro].
- Studying a rare case of a professional musician implanted with intracranial EEG, providing detailed insights into the neural processes underlying speech and music production and perception [Lorenz_2025_Cortex].
Work Package 4: Modulation of Speech Processing by Motor Activity
Activities:
- Investigate how overt motor tracking at different rates (syllabic, word, phrasal) influences speech comprehension under varying acoustic conditions.
Main achievements:
-Motor priming (tapping) influences speech comprehension, and performance in degraded speech is positively correlated with perception-production coupling abilities [Berthault_2024_Cognition, teRietmolen_2025_PRSB].
- Rhythmic tapping synchronized to the lexical rate of speech significantly improves speech perception in noise [teRietmolen_2025_PRSB].
Conclusion:
These results demonstrate that the "SPEEDY" project has made significant advances in understanding the neural processing of speech and its multiscale dynamics, achieving several of the objectives outlined in the initial work packages.
The publications in high-impact journals attest to the quality and importance of these scientific and technical accomplishments.