Periodic Reporting for period 1 - UMMs (Unifying Monitoring Models of Verbal Monitoring.)
Berichtszeitraum: 2019-05-01 bis 2021-04-30
The research project investigates multiple fundamental aspects of the verbal monitoring mechanism. While the reduction of error production, and the ability to resolve errors heard in the speech of someone else, are very central to our ability to communicate, many fundamental aspects of this verbal monitoring mechanism are as of yet unknown.
It is important for society to increase our understanding of the verbal monitoring system, as deficits in the verbal monitoring system have been indicated as a source for multiple debilitating pathologies, such as stammering and hearing voices (e.g. as observed in schizophrenia). By better understanding the healthy verbal monitoring system, we better understand what goes wrong in the case of pathologies. This increased understanding allows us to develop new tools and therapies for those in need.
The objective of the research proposal was to investigate if and how cognitive control and language production skills are related to verbal monitoring behavior, thereby investigating fundamental assumptions of current verbal monitoring models. The revised proposal also revolved around investigating fundamental aspects of verbal monitoring, but in a less dependent and coherent way.
It is important for society to increase our understanding of the verbal monitoring system, as deficits in the verbal monitoring system have been indicated as a source for multiple debilitating pathologies, such as stammering and hearing voices (e.g. as observed in schizophrenia). By better understanding the healthy verbal monitoring system, we better understand what goes wrong in the case of pathologies. This increased understanding allows us to develop new tools and therapies for those in need.
The objective of the research proposal was to investigate if and how cognitive control and language production skills are related to verbal monitoring behavior, thereby investigating fundamental assumptions of current verbal monitoring models. The revised proposal also revolved around investigating fundamental aspects of verbal monitoring, but in a less dependent and coherent way.
The initial proposal investigated how language production skills and cognitive control skills contribute to self-monitoring by re-recruiting participants from large scale studies that have either very good or poor language production skills and very good or poor cognitive control skills. However, due to the COVID-19 pandemic, these large-scale studies and thus the ability to re-recruit the participants came to an abrupt halt. Once it became clear that the pandemic was not over after two lockdowns, we created a novel proposal to investigate fundamental aspects of verbal monitoring without reliance on large cohorts. At the end of 2021the researcher was offered a position at the Institute for Allied Health Professions at the Utrecht University of Applied Sciences as a Research Policy Advisor. The position provides an exciting opportunity to shape the research policy in the Netherlands, and to help improve the research skills of professionals in the allied health professions field. As s a result the Marie Curie research project halted on December 31st 2021. Below I summarize the work packages and progress made up until end of the project.
WP 1. Using the Santa Barbara Corpus (SBC) of Spoken American English we investigate verbal monitoring behaviour in naturalistic speech. We will describe how demographic characteristics (gender, age, years of education) correlate with monitoring behaviour. Additionally, the study investigates how setting (formal/informal) affects monitoring behaviour as per theories of cognitive control.
Progress WP1
From the SBC we started with ten conversations that were listened to and every disfluency and error was coded. After coding we discussed on how to proceed. The sound files from the SBC come from naturalistic, non-experimental settings. As a result, many fragments are very noisy; speakers are eating while they speak, multiple speakers speak at the same time. Furthermore, for an effective analysis a recording of the transcripts was required such that each error and repair would constitute one line in the file- a very time-consuming process. At this point we decided that perhaps a different corpus, like the Switchboard Corpus would be more suitable to investigate interruptions and repairs. A quick initial look identified 25000 errors. This thus seemed a more promising avenue to pursue for research. It was as this point that I was offered the new job. After careful consideration we agreed that I will not be involved in further processing and interpretation of the data after starting my new job.
WP2. Proof of principle study. The perception of hearing voices in the absence of a speaker, also known as Auditory Verbal Hallucination (AVH) has been linked to a defect in the verbal monitoring system. A number of studies from the 70s have demonstrated that during AVH perception using sensitive machinery subvocal speech production can be detected. This study set out to investigate whether modern technological advances in brain signal processing allow to detect this subvocal speech production during AVH within a single participant and in real time. If yes, then this signal can be used to create a biofeedback system which trains the AVH perceiver to stop the subvocal production by engaging in a different task or relaxing the muscles.
Progress WP2
For this study a collaboration was sought with prof. Somemr from the University Medical Centre Groningen (UMCG). Ethical approval was obtained from the local Medical Ethical Committee.
In collaboration with dr. Oostenveld from the host institute the technical side of the project was developed. A portable brain signal amplifier transmitted the signal to one computer where it was recorded. The signal was also transmitted to a second computer where it was displayed such that the person from whom measurement were taken could observe the activity in a meaningful way. This was a considerable feat of engineering. Next a series of pilot studies was planned in which we would examine the best way to identify the threshold and filters to be used when creating the visual feedback in response to the AVH. Once the news of my new position came in we ceased the development of the technical side of the research project. The UMCG group is still enthusiastic and invested in the project and we are looking for avenues that will allow them to continue the development of the research idea.
WP 3. Error detection during speech perception. This study investigates how error repair affects EEG signals that respond to speech errors. By varying the information structure of the sentence, we can test predictions arising from the different speech perception models; whether all incoming speech is processed, or whether the ‘good enough’ approach is used in which processing is task dependent.
Progress WP2
This project was presented in a meeting of the Neurobiology of Language Lab and useful feedback was received which allowed a further refinement of the proposed experiments. No further progress was made on this project.
WP 1. Using the Santa Barbara Corpus (SBC) of Spoken American English we investigate verbal monitoring behaviour in naturalistic speech. We will describe how demographic characteristics (gender, age, years of education) correlate with monitoring behaviour. Additionally, the study investigates how setting (formal/informal) affects monitoring behaviour as per theories of cognitive control.
Progress WP1
From the SBC we started with ten conversations that were listened to and every disfluency and error was coded. After coding we discussed on how to proceed. The sound files from the SBC come from naturalistic, non-experimental settings. As a result, many fragments are very noisy; speakers are eating while they speak, multiple speakers speak at the same time. Furthermore, for an effective analysis a recording of the transcripts was required such that each error and repair would constitute one line in the file- a very time-consuming process. At this point we decided that perhaps a different corpus, like the Switchboard Corpus would be more suitable to investigate interruptions and repairs. A quick initial look identified 25000 errors. This thus seemed a more promising avenue to pursue for research. It was as this point that I was offered the new job. After careful consideration we agreed that I will not be involved in further processing and interpretation of the data after starting my new job.
WP2. Proof of principle study. The perception of hearing voices in the absence of a speaker, also known as Auditory Verbal Hallucination (AVH) has been linked to a defect in the verbal monitoring system. A number of studies from the 70s have demonstrated that during AVH perception using sensitive machinery subvocal speech production can be detected. This study set out to investigate whether modern technological advances in brain signal processing allow to detect this subvocal speech production during AVH within a single participant and in real time. If yes, then this signal can be used to create a biofeedback system which trains the AVH perceiver to stop the subvocal production by engaging in a different task or relaxing the muscles.
Progress WP2
For this study a collaboration was sought with prof. Somemr from the University Medical Centre Groningen (UMCG). Ethical approval was obtained from the local Medical Ethical Committee.
In collaboration with dr. Oostenveld from the host institute the technical side of the project was developed. A portable brain signal amplifier transmitted the signal to one computer where it was recorded. The signal was also transmitted to a second computer where it was displayed such that the person from whom measurement were taken could observe the activity in a meaningful way. This was a considerable feat of engineering. Next a series of pilot studies was planned in which we would examine the best way to identify the threshold and filters to be used when creating the visual feedback in response to the AVH. Once the news of my new position came in we ceased the development of the technical side of the research project. The UMCG group is still enthusiastic and invested in the project and we are looking for avenues that will allow them to continue the development of the research idea.
WP 3. Error detection during speech perception. This study investigates how error repair affects EEG signals that respond to speech errors. By varying the information structure of the sentence, we can test predictions arising from the different speech perception models; whether all incoming speech is processed, or whether the ‘good enough’ approach is used in which processing is task dependent.
Progress WP2
This project was presented in a meeting of the Neurobiology of Language Lab and useful feedback was received which allowed a further refinement of the proposed experiments. No further progress was made on this project.
The results of this research project are negligible. The sudden onset of the pandemic thwarted any possibility of conducting the state-of-the-art initial research proposal. As at the onset of the pandemic it was entirely unclear that it would last this long, considerable time was wasted waiting for the pandemic to resolve and make the research project possible again. Once we made amendments to the research project, none of the work packages progressed beyond initial stages as the researchers has moved on to a career opportunity as Research Policy Advisor at a University for Applied Science.