Audiovisual Speech Segmentation and Oscillations

Objetivo

Daily social tasks (e.g. meetings, shopping) rely on human communication with speech. Auditory only studies established that continuous speech segmentation relies on the temporal integration of rhythmic acoustic features occurring at delta-theta rates in the signal, via a possible mechanism of entrainment coupled with beta activity. In terms of neural correlates, the timing network (including the basal ganglia, supplementary motor area and cerebellum) may actively contribute to the temporal integration of auditory signal during continuous speech segmentation. However, speech is often multimodal and listeners need to integrate speaker’s body movements aligned to rhythmic acoustic features. Consequently, AV speech may generate different rhythmic patterns than auditory only speech. How the brain translates naturally concomitant AV rhythms to facilitate speech segmentation clearly remains to be addressed. The goal of this project is to investigate the oscillatory patterns and neural correlates of AV rhythms resulting from the temporal integration of body movements and salient acoustic features in speech. In a first EEG experiment, I will compare the modulations of delta-theta entrainment and coupling with beta activity, depending on the relationship between visual and auditory information (congruent, incongruent and auditory only). Secondly, in an fMRI version of the experiment, I will explore the contribution of the timing network during AV speech segmentation, and its different patterns of activations across speech conditions. Finally, to test whether the brain areas revealed by fMRI have a necessary role for the successful AV speech segmentation, I will run the same fMRI experiment with Parkinson’s disease patients with dysfunctional basal ganglia. Looking at the timing network patterns of activations will also reveal if correlated body information eventually help PD to compensate basal ganglia deficit with greater contribution of alternative path (i.e. cerebellum).