Periodic Reporting for period 1 - DEMOS (Disfluencies and Eye MOvements during Speech: what can they reveal about language production?)
Reporting period: 2019-05-01 to 2021-04-30
The project aims at better understanding where disfluency comes from. The term ‘disfluency’ includes various phenomena such as filled or silent pauses, repeated words, and self-corrections. Despite the high frequency of disfluencies, the question remains as to why speakers are disfluent. To date, no psycholinguistic model of language production takes into account disfluencies or explains mechanistically why they occur. A systematic investigation of the processing levels in language production is essential to uncover the mapping between disfluency and the specific level at which language difficulty is encountered.
Objective 1: Where does disfluency take place? In other words, which pattern of disfluency occurs depending on lexical selection, conceptual or phonological difficulty?
Additionally, because of this close coupling between speaking and seeing, eye- tracking is a promising tool for language production research, in that the pattern of eye- movements may be indicative of the mechanism underlying the disfluency. In particular, it is known that this pattern differs for correct productions and errors. It is also likely that eye-movements will differ for different kind of disfluencies. A last tool will be used to help answer these questions: neuropsychological tests. Studies of pathological discourse production always include such tests to control for other cognitive processes that would interfere with the task. This can allow us to see whether scores on other cognitive abilities predict particular disfluencies.
Objective 2: Which other problems cause disfluency? Which disfluencies and eye-movements are associated with other mechanisms than difficulties in speech encoding?
Conclusions: the project could help uncover disfluencies related to difficulties occurring a different levels of language production. It also showed that not all disfluencies where related to such difficulties: some reflected other cognitive processes or communicative strategies. Although DEMOS focuses on a fundamental science question, it is a mandatory step towards development of applied research projects in clinical populations characterized by a high rate of disfluencies and communicative impairment, such as ADHD, Alzheimer’s disease, or aphasia.
Objective 1: Where does disfluency take place? In other words, which pattern of disfluency occurs depending on lexical selection, conceptual or phonological difficulty?
Additionally, because of this close coupling between speaking and seeing, eye- tracking is a promising tool for language production research, in that the pattern of eye- movements may be indicative of the mechanism underlying the disfluency. In particular, it is known that this pattern differs for correct productions and errors. It is also likely that eye-movements will differ for different kind of disfluencies. A last tool will be used to help answer these questions: neuropsychological tests. Studies of pathological discourse production always include such tests to control for other cognitive processes that would interfere with the task. This can allow us to see whether scores on other cognitive abilities predict particular disfluencies.
Objective 2: Which other problems cause disfluency? Which disfluencies and eye-movements are associated with other mechanisms than difficulties in speech encoding?
Conclusions: the project could help uncover disfluencies related to difficulties occurring a different levels of language production. It also showed that not all disfluencies where related to such difficulties: some reflected other cognitive processes or communicative strategies. Although DEMOS focuses on a fundamental science question, it is a mandatory step towards development of applied research projects in clinical populations characterized by a high rate of disfluencies and communicative impairment, such as ADHD, Alzheimer’s disease, or aphasia.
The project has achieved most of its objectives and milestones for the period (considering the delay to the COVID-19 pandemic and the “Force Majeure”).
In work-package 1, we used a network task, during which participants describe the route taken by a marker through visually presented networks of objects while their eye-movements were monitored (Figure 1). We manipulated lexical selection difficulty and grammatical selection difficulty. To do so, we used pictures with a high name agreement and pictures with a low name agreement, that had a common gender or neuter gender. We replicated the finding that lexical selection difficulty (low name agreement) induces more pauses and self-corrections while neuter gender did not elicit more disfluency. Eye-tracking helped us demonstrate that not all disfluencies were related to this linguistic manipulation (e.g. some reflected a stalling strategy). Finally, by using statistical methods based on machine learning (multivariate pattern analyses), we found that all participants had a similar behavior when naming a picture with a low name agreement (both in terms of disfluency and eye-movements, Figure 1B). This work-package led to a publication in Language, Cognition and Neuroscience, as well as presentations to five international conferences.
In work-package 2, we asked whether delays in the earliest stages of picture naming elicit disfluency. To address this question, we used visual blurring which hinders visual identification of the items and thereby slows down selection of a lexical concept. We tested the effect of this manipulation on disfluency production and visual attention, similarly to Work-Package 1. Contrary to what was expected, blurriness did not lead to more disfluency on average and viewing times decreased with blurred pictures. However, multivariate pattern analyses revealed that a classifier could predict, from the pattern of disfluency, whether each participant was about to name blurred or control pictures. Impeding the conceptual generation of a message therefore affected the pattern of disfluencies of each participant individually, but this pattern was not consistent from one participant to another. This finding suggests that conceptual difficulty manifests itself differently from one participant to another. Correlations with cognitive performance provided insights into these inter-individual differences, suggesting an influence of inhibition and processing speed on disfluency production. For this study, a paper is currently under review. It also led to two proceedings and two presentations to international conferences (ExLing Society and DiSS workshop).
In work-package 3 (on-going), we investigate the cognitive mechanisms of semantic interference. To do so, we use a word-picture interference paradigm, in which participants have to name a picture with a superimposed word written on it (semantically related or not). Behavioral performance will then modelled using the drift diffusion model, an analytical approach that combines reaction time distributions for correct and incorrect task responses in order to estimate latent variables associated with task performance. This aims at uncovering the origin of such interference: at the lexico-semantic level or in the domain of executive control. In a second step, we will focus more specifically on the mechanisms underlying disfluency in such tasks: lexico-semantic, executive control, or attentional bias.
In work-package 1, we used a network task, during which participants describe the route taken by a marker through visually presented networks of objects while their eye-movements were monitored (Figure 1). We manipulated lexical selection difficulty and grammatical selection difficulty. To do so, we used pictures with a high name agreement and pictures with a low name agreement, that had a common gender or neuter gender. We replicated the finding that lexical selection difficulty (low name agreement) induces more pauses and self-corrections while neuter gender did not elicit more disfluency. Eye-tracking helped us demonstrate that not all disfluencies were related to this linguistic manipulation (e.g. some reflected a stalling strategy). Finally, by using statistical methods based on machine learning (multivariate pattern analyses), we found that all participants had a similar behavior when naming a picture with a low name agreement (both in terms of disfluency and eye-movements, Figure 1B). This work-package led to a publication in Language, Cognition and Neuroscience, as well as presentations to five international conferences.
In work-package 2, we asked whether delays in the earliest stages of picture naming elicit disfluency. To address this question, we used visual blurring which hinders visual identification of the items and thereby slows down selection of a lexical concept. We tested the effect of this manipulation on disfluency production and visual attention, similarly to Work-Package 1. Contrary to what was expected, blurriness did not lead to more disfluency on average and viewing times decreased with blurred pictures. However, multivariate pattern analyses revealed that a classifier could predict, from the pattern of disfluency, whether each participant was about to name blurred or control pictures. Impeding the conceptual generation of a message therefore affected the pattern of disfluencies of each participant individually, but this pattern was not consistent from one participant to another. This finding suggests that conceptual difficulty manifests itself differently from one participant to another. Correlations with cognitive performance provided insights into these inter-individual differences, suggesting an influence of inhibition and processing speed on disfluency production. For this study, a paper is currently under review. It also led to two proceedings and two presentations to international conferences (ExLing Society and DiSS workshop).
In work-package 3 (on-going), we investigate the cognitive mechanisms of semantic interference. To do so, we use a word-picture interference paradigm, in which participants have to name a picture with a superimposed word written on it (semantically related or not). Behavioral performance will then modelled using the drift diffusion model, an analytical approach that combines reaction time distributions for correct and incorrect task responses in order to estimate latent variables associated with task performance. This aims at uncovering the origin of such interference: at the lexico-semantic level or in the domain of executive control. In a second step, we will focus more specifically on the mechanisms underlying disfluency in such tasks: lexico-semantic, executive control, or attentional bias.
This project allowed a broad and pioneering investigation of language and disfluency: first, by tackling the different levels of the production system using a consistent taxonomy; second, by developing an approach that aims to disentangle the underlying cause of each disfluency (i.e. due to difficulty at a specific level of the production system or taking place outside the language production system). Additionally, this project will develop a novel use of eye-tracking, which will generate conceptual and empirical insights that will be beneficial for eye-tracking and speech production communities, as well as clinical language science. This project will also be beneficial for the community of researchers who study language production since written transcripts and audio material collected were made freely accessible on the FluencyBank database after ensuring participants agreement. This database is part of TalkBank, an international system for the exchange of data on spoken language interactions. Since disfluencies are indeed present in many diseases (e.g. Alzheimer's disease, ADHD, stuttering, etc.), findings will allow a better understanding of patients' communicative impairment. More generally, current findings will promote the use of novel tasks in clinical settings, to test language difficulties. There are no direct socio-economic impact, however, current findings can help patients' diagnosis and assessment in an effective and inexpensive way.