Fundamentals of formal properties of nonmanuals: A quantitative approach

Informations projet

NONMANUAL

N° de convention de subvention: 101039378

DOI

10.3030/101039378

Date de signature de la CE 28 Avril 2022

Date de début 1 Janvier 2023

Date de fin 31 Decembre 2027

Financé au titre de

• European Research Council (ERC)

Coût total

€ 1 500 000,00

Contribution de l’UE

€ 1 500 000,00

Coordonné par

UNIVERSITETET I BERGEN
Norway

Periodic Reporting for period 1 - NONMANUAL (Fundamentals of formal properties of nonmanuals: A quantitative approach)

Période du rapport: 2023-01-01 au 2025-06-30

Résumé du contexte et des objectifs généraux du projet

Sign languages, in addition to using the hands, also use positions and movements of other articulators: the body, the head, the mouth, the eyebrows, the eyes and the eyelids, to convey lexical, grammatical, and prosodic information. This linguistic use of the nonmanual articulators is known as nonmanuals. Contrary to current assumptions in the field of sign linguistics, this project proposes the hypothesis that all sign languages use the same basic universal building blocks (nonmanual movements) but that each language is different in how it combines these building blocks both sequentially and simultaneously; languages also differ in the regularity, frequency, and the alignment properties of the nonmanuals. In order to test this hypothesis, we use Computer Vision to measure nonmanuals and advanced statistical techniques to analyze the formal properties of nonmanuals and compare them across contexts and languages. Using these techniques, we analyze naturalistic corpus data (recordings of natural conversations) in several sign languages, and focus on a variety of nonmanuals, such as head shakes used to express negation, head movements marking questions, and eye blinks marking sentence boundaries. We want to find out the extent of variation between the languages in how nonmanuals are used, and the universal patterns undelrying this variation.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

In the first two years of the project, we pursued several research directions.

On the technical side, we have tested and calibrated several Computer Vision solutions, specifically for measuring head movements and detecting eyeblinks. In addition, we have collected a naturalistic data set of Norwegian Sign Language, in which 10 deaf signers produce sentences with various nonmanuals while being recorded by multiple cameras (including depth measuring cameras). We will use this data set for further testing and improvement of Computer Vision instruments. Finally, we have been working on statistical solutions for the analysis of the measurements produced by the instruments. We are developing procedures for dealing with noisy data, detecting the relevant shapes on nonmanuals, and producing various kinematic measurements based on these measurements.

On the linguistic side, we have been quantitatively analyzing various nonmanuals across different sign languages. For example, we analyzed head shakes expressing negation in Sign Language of the Netherlands (NGT). We discovered that kinematic properties of the head shakes (such as duration, amplitude and velocity) can be affected by linguistic factors, such as whether the headshake spreads across several manual signs, and which signs it spreads over. This showcases our approach: it is possible to quantitatively investigate the formal properties of nonmanuals and how they are affected by linguistic features within and across languages. We are currently working on a larger scale comparison of head shake across several sign languages.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The current project is among the first studies to showcase the possibility of conducting qunatitative research on nonmanuals using Computer Vision. While the initial results mainly concern singular sign languages and specific nonmanuals, it shows the potential to conduct large scale comparative research on nonmanuals to discover language-specific and universal patterns in the shapes on nonmanuals.

Visualizatio of a head shake in Sign Language of the Netherlands