Final Report Summary - SPEECH UNIT(E)S (The multisensory-motor unity of speech)
The Speech Unit(e)s project was focused on the speech unification process associating the auditory, visual and motor streams in the human brain, in an interdisciplinary approach combining cognitive psychology, neurosciences, phonetics (both descriptive and developmental) and computational models. The framework was provided by the “Perception-for-Action-Control Theory (PACT)”, a perceptuo-motor theory of speech communication, considering that speech units are neither a sound nor a gesture but a perceptually shaped gesture, that is a perceptuo-motor unit, characterized by both articulatory coherence – provided by its gestural nature – and perceptual value – necessary for being functional.
This project provided important progress in three major directions.
Firstly, it led to a number of experimental findings on the role of perceptuo-motor relationships in speech acquisition, perception and control, showing e.g. how perceptual and motor knowledge are related in the course of speech development; how auditory, visual and somatosensory information can be bound together to enable formation and use of speech units in speech perception and production; and how sensory or motor perturbations or handicaps could be dealt with efficiently in such a perceptuo-motor framework.
Secondly, an integrative computational model of speech communication, associating speech perception and speech production in a developmental perspective, was elaborated and developed extensively based on concepts and methods coming from Bayesian Algorithmic Modeling. This model, globally called COSMO (Communication Objects using Sensory-Motor Operations) with a number of variants to adapt to various types of situations, enabled to describe and simulate experimental data related to phoneme and syllable perception and production, sensory-motor perturbations, sensory-motor idiosyncrasies, speech acquisition and brain circuits.
Thirdly, experimental data and computational modelling were related to the question of the relationships between phonetics and phonology, with the aim to foster the analysis of phonetic processes and to possibly contribute to develop phonology theories in relationships with neurocognitive data on sensory-motor processes in the human brain.
This project provided important progress in three major directions.
Firstly, it led to a number of experimental findings on the role of perceptuo-motor relationships in speech acquisition, perception and control, showing e.g. how perceptual and motor knowledge are related in the course of speech development; how auditory, visual and somatosensory information can be bound together to enable formation and use of speech units in speech perception and production; and how sensory or motor perturbations or handicaps could be dealt with efficiently in such a perceptuo-motor framework.
Secondly, an integrative computational model of speech communication, associating speech perception and speech production in a developmental perspective, was elaborated and developed extensively based on concepts and methods coming from Bayesian Algorithmic Modeling. This model, globally called COSMO (Communication Objects using Sensory-Motor Operations) with a number of variants to adapt to various types of situations, enabled to describe and simulate experimental data related to phoneme and syllable perception and production, sensory-motor perturbations, sensory-motor idiosyncrasies, speech acquisition and brain circuits.
Thirdly, experimental data and computational modelling were related to the question of the relationships between phonetics and phonology, with the aim to foster the analysis of phonetic processes and to possibly contribute to develop phonology theories in relationships with neurocognitive data on sensory-motor processes in the human brain.