Final Report Summary - GECZENGLOTT (Glottalizations in German and Czech English)
1) Summary description of the project objectives:
The project objectives are to analyse: 1) the extent of glottalization before word-initial vowels in German English in relationship to prosodic structure and to compare it with Czech English and British English, 2) the influence of glottalizations on word-recognition by German subjects in comparison to Czech and native English subjects, and 3) the influence of glottalizations on the perception of foreign accent by native English speakers.
Glottalizations are sounds produced by the closing and abrupt opening of the vocal folds. In German and Czech, glottalizations are frequent before word-initial vowels and are a salient word-linking technique. In English glottalizations are less frequent. Dr. Bissiri’s previous analyses have shown that Czech speakers transfer this word-linking habit to their English productions and that glottalizations influence word recognition differently for Czech, English and Spanish subjects.
2) Description of the work performed since the beginning of the project:
At the beginning of the project a speech database of German-accented English was collected. As a first step in the analysis, automatic speech recognition experiments with the recorded signals were carried out by means of the Unified Approach to Speech Synthesis and Recognition (UASR) System developed at the Institute of Acoustics and Speech Communication (IAS) of the Technical University of Dresden, the host organization. The experiments tested the automatic recognition of German-accented English, especially of glottalizations, and produced an automatic labeling of the data. Word-initial glottalizations were manually labelled in the data. In order to evalutate the automatic speech recognition performance, the automatic labels were compared to the manual labels. Subsequently, prosodic boundaries were manually labeled in the signals. Then, the extent of glottalization in relationship to phrase boundaries was analyzed in the German-accented English data in comparison to Czech English and British English data.
Several methods for resynthesizing glottalization were then investigated, with the purpose of creating stimuli for the planned speech perception experiments. The most natural stimuli were those created by splicing natural glottalizations into them, thus this method was chosen for creating the stimuli for the perceptual experiments in the present project.
Subsequent experiments investigated the perception of pitch in glottalization by native German and native English speakers. The stimuli were recorded in studio in modal and glottalized version and pitch was manipulated by means of the software Praat. Experiments on the perception of glottalization in different pitch environments were carried out with German and English native speakers. Given the linguistic relevance of the investigation and the necessity to verify results with other languages, experiments were carried out also with Swedish natives, in collaboration with the Royal Institute of Technology in Stockholm, with Chinese natives at the Tongji University of Shanghai and with Macedonian natives at the Ss. Cyril and Methodius University of Skopje.
Further experiments were carried out on the analysis and synthesis of glottalization phenomena in the German-accented English corpus. For each glottalized segment, time-normalized F0 and log-energy contours were produced and principal component analysis was performed on the contour sets in order to reduce their dimensionality. Centroid contours of the PC clusters were used for contour reconstruction in a resynthesis experiments. The prototype intonation and intensity contours were superimposed over non-glottalized word-initial vowels in order to resynthesize creaky voice. This procedure allowed the automatic creation of glottalized speech stimuli.
A more suitable method than word recognition was employed to investigate the perception of glottalization. A perceptual experiment was set up to test if frequent word-initial glottalizations in the native language (as in German), as opposed to less frequent word-initial glottalizations (as in English) lead to a different perception of glottalization, as a word-initial marker rather than as intonation (falling pitch). For this purpose, stimuli were created with a 10-step continuum from a glottal stop followed by a modal vowel, until a glottal stop followed by a complete creaky vowel. The experiment was carried out with German, English and Swedish native listeners. The experiments with the Swedish native listeners were carried out in collaboration with the Royal Institute of Technology in Stockholm. Perceptual data were evaluated by means of logistic regression analysis.
3) Description of the main results achieved:
The experiments with automatic speech recognition provided a reliable phoneme and word segmentation framework, thus a valuable method for the pre-segmentation of German-accented English for the purpose of phonetic analysis could be obtained. Glottal stops were positively identified by the ASR system with 93% accuracy. Automatic boundaries of glottal stops have been placed with reasonable precision, being the mean shifts from the manual ones between 5 and 20 ms.
The investigation of the extent of glottalization before word-initial vowels in German-accented English productions in comparison with Czech English and British English showed that Germans, as Czech speakers, transfer their habit of frequently glottalizing at word-boundaries from their native language to their English productions. In German-accented English, a high percentage of the tokens are glottalized and creaky voice is found as the most frequently produced glottalization type, in line with previous studies on native German, while glottal stops are more frequent in Czech as well as in Czech-accented English. In German-accented English, glottalizations are more frequent at phrase boundaries than at non-phrase boundaries. German speakers with longer experience in an English speaking environment display lower glottalization frequencies than the other German speakers, but higher than English natives. This could mean that the transfer of word-linking habits from the native language is a persistent characteristic of German-accented English.
Results of the experiments on pitch perception in glottalization show that although F0 within glottalization can be perceived as pitch, it need not be, and that linguistic experience can influence the degree to which listeners listen for pitch information or respond to glottalization. In the perceptual experiments with stimuli consisting in a continuum from modal to creaky voice, the German listeners associated longer stretches of glottalization to falling pitch less frequently than English listeners. This is possibly due to a different phonological interpretation of glottalizations by German and by English listeners. Since in German word-initial glottalization is more frequent, German interpreted glottalization in the stimuli as word-initial, while English natives considered it as intonation, thus associating it more frequently to falling pitch. This would also explain why English natives gave more fall responses when initial pitch was a rise, as in the previous experiments, while German natives did not.
Regarding the manipulation of glottalization in order to create stimuli for perception experiments, several attempts have been carried out to remove/add glottalizations from/to speech signals. The software used were Praat, STRAIGHT and the dLabPro toolkit with the UASR system. A good automatic procedure was developed to add glottalization to modal stretches of speech by means of prosody contour templates created by statistical and Principal Component Analysis.
4) Potential impact and use of the results (including the socio-economic impact and the wider societal implications of the project):
The project results implicate that glottalizations are a relevant and complex phenomenon which should be explicitly addressed in language teaching and training. Not only are glottalizations actively transferred from the native language to second language productions, but they also influence speech perception and the phonological interpretation of speech by native speakers of other languages, such as Swedish, Chinese, and Macedonian.
The results of the present project build the base of further investigations which Dr. Bissiri will carry out in a following Marie Curie Project at the Queen Margaret University in Edinburgh. Dr. Bissiri will carry out also articulatory analysis on glottalization and related phenomena, and will further investigate the perception of glottalization in relationship with other word-boundary phenomena.
The project results delivered an effective method to automatically generate a pre-segmentation for phonetic research. Furthermore, the experiments on the authomatic analysis and synthesis of glottalizations could create naturally sounding stimuli with manipulated glottalization. Such procedures can be effectively employed for further investigations on the production and perception of glottalizations.
The project created and/or strenghtened international research collaborations, e.g. of the host institution with the Royal Institute of Technology in Stockholm, the Tongji University in Shanghai, the Queen Margaret University in Edinburgh, and the Ss. Cyril and Methodius University of Skopj. These collaborations will have a long-term positive effect.
Moreover, research on English as second language is a quite relevant and useful topic for today’s need of learning English. This fact makes research on second language particularly attractive for researchers and fundings, thus contributing to European excellence and competitiveness.
Project website: http://wwwpub.zih.tu-dresden.de/~bissiri/GeCzEnGlott.htm
The project objectives are to analyse: 1) the extent of glottalization before word-initial vowels in German English in relationship to prosodic structure and to compare it with Czech English and British English, 2) the influence of glottalizations on word-recognition by German subjects in comparison to Czech and native English subjects, and 3) the influence of glottalizations on the perception of foreign accent by native English speakers.
Glottalizations are sounds produced by the closing and abrupt opening of the vocal folds. In German and Czech, glottalizations are frequent before word-initial vowels and are a salient word-linking technique. In English glottalizations are less frequent. Dr. Bissiri’s previous analyses have shown that Czech speakers transfer this word-linking habit to their English productions and that glottalizations influence word recognition differently for Czech, English and Spanish subjects.
2) Description of the work performed since the beginning of the project:
At the beginning of the project a speech database of German-accented English was collected. As a first step in the analysis, automatic speech recognition experiments with the recorded signals were carried out by means of the Unified Approach to Speech Synthesis and Recognition (UASR) System developed at the Institute of Acoustics and Speech Communication (IAS) of the Technical University of Dresden, the host organization. The experiments tested the automatic recognition of German-accented English, especially of glottalizations, and produced an automatic labeling of the data. Word-initial glottalizations were manually labelled in the data. In order to evalutate the automatic speech recognition performance, the automatic labels were compared to the manual labels. Subsequently, prosodic boundaries were manually labeled in the signals. Then, the extent of glottalization in relationship to phrase boundaries was analyzed in the German-accented English data in comparison to Czech English and British English data.
Several methods for resynthesizing glottalization were then investigated, with the purpose of creating stimuli for the planned speech perception experiments. The most natural stimuli were those created by splicing natural glottalizations into them, thus this method was chosen for creating the stimuli for the perceptual experiments in the present project.
Subsequent experiments investigated the perception of pitch in glottalization by native German and native English speakers. The stimuli were recorded in studio in modal and glottalized version and pitch was manipulated by means of the software Praat. Experiments on the perception of glottalization in different pitch environments were carried out with German and English native speakers. Given the linguistic relevance of the investigation and the necessity to verify results with other languages, experiments were carried out also with Swedish natives, in collaboration with the Royal Institute of Technology in Stockholm, with Chinese natives at the Tongji University of Shanghai and with Macedonian natives at the Ss. Cyril and Methodius University of Skopje.
Further experiments were carried out on the analysis and synthesis of glottalization phenomena in the German-accented English corpus. For each glottalized segment, time-normalized F0 and log-energy contours were produced and principal component analysis was performed on the contour sets in order to reduce their dimensionality. Centroid contours of the PC clusters were used for contour reconstruction in a resynthesis experiments. The prototype intonation and intensity contours were superimposed over non-glottalized word-initial vowels in order to resynthesize creaky voice. This procedure allowed the automatic creation of glottalized speech stimuli.
A more suitable method than word recognition was employed to investigate the perception of glottalization. A perceptual experiment was set up to test if frequent word-initial glottalizations in the native language (as in German), as opposed to less frequent word-initial glottalizations (as in English) lead to a different perception of glottalization, as a word-initial marker rather than as intonation (falling pitch). For this purpose, stimuli were created with a 10-step continuum from a glottal stop followed by a modal vowel, until a glottal stop followed by a complete creaky vowel. The experiment was carried out with German, English and Swedish native listeners. The experiments with the Swedish native listeners were carried out in collaboration with the Royal Institute of Technology in Stockholm. Perceptual data were evaluated by means of logistic regression analysis.
3) Description of the main results achieved:
The experiments with automatic speech recognition provided a reliable phoneme and word segmentation framework, thus a valuable method for the pre-segmentation of German-accented English for the purpose of phonetic analysis could be obtained. Glottal stops were positively identified by the ASR system with 93% accuracy. Automatic boundaries of glottal stops have been placed with reasonable precision, being the mean shifts from the manual ones between 5 and 20 ms.
The investigation of the extent of glottalization before word-initial vowels in German-accented English productions in comparison with Czech English and British English showed that Germans, as Czech speakers, transfer their habit of frequently glottalizing at word-boundaries from their native language to their English productions. In German-accented English, a high percentage of the tokens are glottalized and creaky voice is found as the most frequently produced glottalization type, in line with previous studies on native German, while glottal stops are more frequent in Czech as well as in Czech-accented English. In German-accented English, glottalizations are more frequent at phrase boundaries than at non-phrase boundaries. German speakers with longer experience in an English speaking environment display lower glottalization frequencies than the other German speakers, but higher than English natives. This could mean that the transfer of word-linking habits from the native language is a persistent characteristic of German-accented English.
Results of the experiments on pitch perception in glottalization show that although F0 within glottalization can be perceived as pitch, it need not be, and that linguistic experience can influence the degree to which listeners listen for pitch information or respond to glottalization. In the perceptual experiments with stimuli consisting in a continuum from modal to creaky voice, the German listeners associated longer stretches of glottalization to falling pitch less frequently than English listeners. This is possibly due to a different phonological interpretation of glottalizations by German and by English listeners. Since in German word-initial glottalization is more frequent, German interpreted glottalization in the stimuli as word-initial, while English natives considered it as intonation, thus associating it more frequently to falling pitch. This would also explain why English natives gave more fall responses when initial pitch was a rise, as in the previous experiments, while German natives did not.
Regarding the manipulation of glottalization in order to create stimuli for perception experiments, several attempts have been carried out to remove/add glottalizations from/to speech signals. The software used were Praat, STRAIGHT and the dLabPro toolkit with the UASR system. A good automatic procedure was developed to add glottalization to modal stretches of speech by means of prosody contour templates created by statistical and Principal Component Analysis.
4) Potential impact and use of the results (including the socio-economic impact and the wider societal implications of the project):
The project results implicate that glottalizations are a relevant and complex phenomenon which should be explicitly addressed in language teaching and training. Not only are glottalizations actively transferred from the native language to second language productions, but they also influence speech perception and the phonological interpretation of speech by native speakers of other languages, such as Swedish, Chinese, and Macedonian.
The results of the present project build the base of further investigations which Dr. Bissiri will carry out in a following Marie Curie Project at the Queen Margaret University in Edinburgh. Dr. Bissiri will carry out also articulatory analysis on glottalization and related phenomena, and will further investigate the perception of glottalization in relationship with other word-boundary phenomena.
The project results delivered an effective method to automatically generate a pre-segmentation for phonetic research. Furthermore, the experiments on the authomatic analysis and synthesis of glottalizations could create naturally sounding stimuli with manipulated glottalization. Such procedures can be effectively employed for further investigations on the production and perception of glottalizations.
The project created and/or strenghtened international research collaborations, e.g. of the host institution with the Royal Institute of Technology in Stockholm, the Tongji University in Shanghai, the Queen Margaret University in Edinburgh, and the Ss. Cyril and Methodius University of Skopj. These collaborations will have a long-term positive effect.
Moreover, research on English as second language is a quite relevant and useful topic for today’s need of learning English. This fact makes research on second language particularly attractive for researchers and fundings, thus contributing to European excellence and competitiveness.
Project website: http://wwwpub.zih.tu-dresden.de/~bissiri/GeCzEnGlott.htm