Periodic Reporting for period 1 - NONMANUAL (Fundamentals of formal properties of nonmanuals: A quantitative approach)
Reporting period: 2023-01-01 to 2025-06-30
Sign languages, in addition to using the hands, also use positions and movements of other articulators: the body, the head, the mouth, the eyebrows, the eyes and the eyelids, to convey lexical, grammatical, and prosodic information. This linguistic use of the nonmanual articulators is known as nonmanuals. Contrary to current assumptions in the field of sign linguistics, this project proposes the hypothesis that all sign languages use the same basic universal building blocks (nonmanual movements) but that each language is different in how it combines these building blocks both sequentially and simultaneously; languages also differ in the regularity, frequency, and the alignment properties of the nonmanuals. In order to test this hypothesis, we use Computer Vision to measure nonmanuals and advanced statistical techniques to analyze the formal properties of nonmanuals and compare them across contexts and languages. Using these techniques, we analyze naturalistic corpus data (recordings of natural conversations) in several sign languages, and focus on a variety of nonmanuals, such as head shakes used to express negation, head movements marking questions, and eye blinks marking sentence boundaries. We want to find out the extent of variation between the languages in how nonmanuals are used, and the universal patterns undelrying this variation.
In the first two years of the project, we pursued several research directions.
On the technical side, we have tested and calibrated several Computer Vision solutions, specifically for measuring head movements and detecting eyeblinks. In addition, we have collected a naturalistic data set of Norwegian Sign Language, in which 10 deaf signers produce sentences with various nonmanuals while being recorded by multiple cameras (including depth measuring cameras). We will use this data set for further testing and improvement of Computer Vision instruments. Finally, we have been working on statistical solutions for the analysis of the measurements produced by the instruments. We are developing procedures for dealing with noisy data, detecting the relevant shapes on nonmanuals, and producing various kinematic measurements based on these measurements.
On the linguistic side, we have been quantitatively analyzing various nonmanuals across different sign languages. For example, we analyzed head shakes expressing negation in Sign Language of the Netherlands (NGT). We discovered that kinematic properties of the head shakes (such as duration, amplitude and velocity) can be affected by linguistic factors, such as whether the headshake spreads across several manual signs, and which signs it spreads over. This showcases our approach: it is possible to quantitatively investigate the formal properties of nonmanuals and how they are affected by linguistic features within and across languages. We are currently working on a larger scale comparison of head shake across several sign languages.
On the technical side, we have tested and calibrated several Computer Vision solutions, specifically for measuring head movements and detecting eyeblinks. In addition, we have collected a naturalistic data set of Norwegian Sign Language, in which 10 deaf signers produce sentences with various nonmanuals while being recorded by multiple cameras (including depth measuring cameras). We will use this data set for further testing and improvement of Computer Vision instruments. Finally, we have been working on statistical solutions for the analysis of the measurements produced by the instruments. We are developing procedures for dealing with noisy data, detecting the relevant shapes on nonmanuals, and producing various kinematic measurements based on these measurements.
On the linguistic side, we have been quantitatively analyzing various nonmanuals across different sign languages. For example, we analyzed head shakes expressing negation in Sign Language of the Netherlands (NGT). We discovered that kinematic properties of the head shakes (such as duration, amplitude and velocity) can be affected by linguistic factors, such as whether the headshake spreads across several manual signs, and which signs it spreads over. This showcases our approach: it is possible to quantitatively investigate the formal properties of nonmanuals and how they are affected by linguistic features within and across languages. We are currently working on a larger scale comparison of head shake across several sign languages.
The current project is among the first studies to showcase the possibility of conducting qunatitative research on nonmanuals using Computer Vision. While the initial results mainly concern singular sign languages and specific nonmanuals, it shows the potential to conduct large scale comparative research on nonmanuals to discover language-specific and universal patterns in the shapes on nonmanuals.