Periodic Reporting for period 2 - PlanArt (Planning the Articulation of Spoken Utterances)
Okres sprawozdawczy: 2023-07-01 do 2024-12-31
The issue addressed in this project is the nature of phonological representations, output goals, and processes used in planning and producing speech articulation. Answers to these questions have implications for our understanding of the basic architecture of the speech production planning process, and are therefore relevant for understanding normal and disordered speech production, as well as speech sound acquisition and speech technology.
The overall project objectives are to provide converging evidence for representations, goals and processes used in planning speech production 1) via experimental tests of predictions for patterns of speech articulation of alternative theories, and 2) via a comparison of articulatory predictions of computational models based on particular (sets of) core theoretical assumptions.
The experiments provide tests of a) symbolic vs. spatiotemporal phonological representations, b) separate phonological and phonetic planning processing components, c) acoustic vs. articulatory goals, d) the nature of speech movement coordination, d) separate representation of spatial vs. temporal information vs. integrated spatiotemporal representations, and e) Lee's General Tau theory of movement planning, coordination, and generation. General Tau Theory has strong support in the non-speech motor literature, but is new to the field of speech science.
The computational modeling work includes 1) the development of a flexible, modular platform allowing for parallel implementation, testing, and comparison of competing theoretical assumptions in terms of
model outputs, and 2) the development of a computational implementation of new approach to speech articulation planning based on symbolic phonological representations, phonology-extrinsic timing, as well as separate Phonological Planning, Phonetic Planning, and Motor-Sensory Implementation components, proposed by Turk & Shattuck-Hufnagel in their Phonology-Extrinsic Timing, 3 Component approach (XT/3C). Key features of Phonetic Planning in this new approach are 1) the use of General Tau Theory in modeling the time-course of articulatory movements, which presents a departure from alternative state-of-the art dynamic models based on mass-spring systems, and 2) the reliance on Optimal Control Theory for accounting for systematic phonetic variability due to competing task demands.
The overall project objectives are to provide converging evidence for representations, goals and processes used in planning speech production 1) via experimental tests of predictions for patterns of speech articulation of alternative theories, and 2) via a comparison of articulatory predictions of computational models based on particular (sets of) core theoretical assumptions.
The experiments provide tests of a) symbolic vs. spatiotemporal phonological representations, b) separate phonological and phonetic planning processing components, c) acoustic vs. articulatory goals, d) the nature of speech movement coordination, d) separate representation of spatial vs. temporal information vs. integrated spatiotemporal representations, and e) Lee's General Tau theory of movement planning, coordination, and generation. General Tau Theory has strong support in the non-speech motor literature, but is new to the field of speech science.
The computational modeling work includes 1) the development of a flexible, modular platform allowing for parallel implementation, testing, and comparison of competing theoretical assumptions in terms of
model outputs, and 2) the development of a computational implementation of new approach to speech articulation planning based on symbolic phonological representations, phonology-extrinsic timing, as well as separate Phonological Planning, Phonetic Planning, and Motor-Sensory Implementation components, proposed by Turk & Shattuck-Hufnagel in their Phonology-Extrinsic Timing, 3 Component approach (XT/3C). Key features of Phonetic Planning in this new approach are 1) the use of General Tau Theory in modeling the time-course of articulatory movements, which presents a departure from alternative state-of-the art dynamic models based on mass-spring systems, and 2) the reliance on Optimal Control Theory for accounting for systematic phonetic variability due to competing task demands.
We have collected data from 24 experimental sessions and are in the process of analysing the data. We have produced modeling software available to the general public: https://git.ecdf.ed.ac.uk/belie/planart(odnośnik otworzy się w nowym oknie)
1) We have provided support for a model of speech production based on symbolic phonological representations that includes separate Phonological and Phonetic Planning Components. This has been shown via
-a) Measurements of behavioural data which show lower timing variability at lower timing variability at goal-related parts of movement trajectories (Turk et al. 2024 Labphon satellite talk). This evidence suggests that symbolic representations map onto the goal-related part of movement so that it can be prioritized for accurate coordination with a reference event. These findings are difficult to explain in models with spatiotemporal representations because the movement endpoint is not represented separately from other parts of movement; the timing of this part of movement therefore can’t be used for coordination, nor can it be separately prioritized.
-b) A demonstration of the superiority of Lee's Tau theory over existing dynamic models in describing real speech movement trajectory data (Elie et al. 2023 Speech Communication). This supports a model of speech production based on symbolic representations and separate Phonological and Phonetic planning components because Tau theory requires a representation of the goal that is separate from the movement(s) that achieve(s) the goal.
-c) Successful modeling of a Phonetic Planning component based on General Tau Theory and Optimal Control Theory that fits in such a model. (Elie et al. 2023 INTERSPEECH; Elie et al. (2024) Speech Communication). This shows the viability of the modeling approach.
2) We have shown the viability of Optimal Control Theory approaches for modeling systematic articulatory variability in speech production: Modeling variation related to speech rate, centralisation of unstressed vowels, lenition of stop consonants in unstressed environments (Elie et al. 2024 Speech Communication), as well as articulatory properties of Lombard speech (Elie et al. 2024 JASA Express Letters).
3) We have shown the usefulness of Lee's Tau theory for speech movement trajectory analysis. Elie & Turk (2023) INTERSPEECH; Turk et al. (2024) BAAP, ISSP.
1) We have provided support for a model of speech production based on symbolic phonological representations that includes separate Phonological and Phonetic Planning Components. This has been shown via
-a) Measurements of behavioural data which show lower timing variability at lower timing variability at goal-related parts of movement trajectories (Turk et al. 2024 Labphon satellite talk). This evidence suggests that symbolic representations map onto the goal-related part of movement so that it can be prioritized for accurate coordination with a reference event. These findings are difficult to explain in models with spatiotemporal representations because the movement endpoint is not represented separately from other parts of movement; the timing of this part of movement therefore can’t be used for coordination, nor can it be separately prioritized.
-b) A demonstration of the superiority of Lee's Tau theory over existing dynamic models in describing real speech movement trajectory data (Elie et al. 2023 Speech Communication). This supports a model of speech production based on symbolic representations and separate Phonological and Phonetic planning components because Tau theory requires a representation of the goal that is separate from the movement(s) that achieve(s) the goal.
-c) Successful modeling of a Phonetic Planning component based on General Tau Theory and Optimal Control Theory that fits in such a model. (Elie et al. 2023 INTERSPEECH; Elie et al. (2024) Speech Communication). This shows the viability of the modeling approach.
2) We have shown the viability of Optimal Control Theory approaches for modeling systematic articulatory variability in speech production: Modeling variation related to speech rate, centralisation of unstressed vowels, lenition of stop consonants in unstressed environments (Elie et al. 2024 Speech Communication), as well as articulatory properties of Lombard speech (Elie et al. 2024 JASA Express Letters).
3) We have shown the usefulness of Lee's Tau theory for speech movement trajectory analysis. Elie & Turk (2023) INTERSPEECH; Turk et al. (2024) BAAP, ISSP.
As analysis of our experimental data proceeds, we expect to provide further behavioural evidence for the nature of phonological representations, the number of processing components, goals of speech production (articulatory vs. acoustic), representations for articulatory movements (spatiotemporal vs. separate representations for temporal vs. spatial aspects of movement), and the nature of movement coordination (based on movement onsets vs. goal-related parts of movement, e.g. endpoints).
We expect to provide the following components in a computational model of XT/3C, to supplement our existing Phonetic Planning component: 1) a Phonological Planning component that assumes the use of symbolic phonological representations and 2) a model of Motor Sensory Implementation that can account for observations of adaptation to altered auditory feedback. Where possible, we will compare the capabilities of these components with other available models.
We expect to provide the following components in a computational model of XT/3C, to supplement our existing Phonetic Planning component: 1) a Phonological Planning component that assumes the use of symbolic phonological representations and 2) a model of Motor Sensory Implementation that can account for observations of adaptation to altered auditory feedback. Where possible, we will compare the capabilities of these components with other available models.