Fundamentals of formal properties of nonmanuals: A quantitative approach

Informazioni relative al progetto

NONMANUAL

ID dell’accordo di sovvenzione: 101039378

DOI

10.3030/101039378

Data della firma CE 28 Aprile 2022

Data di avvio 1 Gennaio 2023

Data di completamento 31 Dicembre 2027

Finanziato da

• European Research Council (ERC)

Costo totale

€ 1 500 000,00

Contributo UE

€ 1 500 000,00

Coordinato da

UNIVERSITETET I BERGEN
Norway

Periodic Reporting for period 1 - NONMANUAL (Fundamentals of formal properties of nonmanuals: A quantitative approach)

Periodo di rendicontazione: 2023-01-01 al 2025-06-30

Sintesi del contesto e degli obiettivi generali del progetto

Sign languages, in addition to using the hands, also use positions and movements of other articulators: the body, the head, the mouth, the eyebrows, the eyes and the eyelids, to convey lexical, grammatical, and prosodic information. This linguistic use of the nonmanual articulators is known as nonmanuals. Contrary to current assumptions in the field of sign linguistics, this project proposes the hypothesis that all sign languages use the same basic universal building blocks (nonmanual movements) but that each language is different in how it combines these building blocks both sequentially and simultaneously; languages also differ in the regularity, frequency, and the alignment properties of the nonmanuals. In order to test this hypothesis, we use Computer Vision to measure nonmanuals and advanced statistical techniques to analyze the formal properties of nonmanuals and compare them across contexts and languages. Using these techniques, we analyze naturalistic corpus data (recordings of natural conversations) in several sign languages, and focus on a variety of nonmanuals, such as head shakes used to express negation, head movements marking questions, and eye blinks marking sentence boundaries. We want to find out the extent of variation between the languages in how nonmanuals are used, and the universal patterns undelrying this variation.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

In the first two years of the project, we pursued several research directions.

On the technical side, we have tested and calibrated several Computer Vision solutions, specifically for measuring head movements and detecting eyeblinks. In addition, we have collected a naturalistic data set of Norwegian Sign Language, in which 10 deaf signers produce sentences with various nonmanuals while being recorded by multiple cameras (including depth measuring cameras). We will use this data set for further testing and improvement of Computer Vision instruments. Finally, we have been working on statistical solutions for the analysis of the measurements produced by the instruments. We are developing procedures for dealing with noisy data, detecting the relevant shapes on nonmanuals, and producing various kinematic measurements based on these measurements.

On the linguistic side, we have been quantitatively analyzing various nonmanuals across different sign languages. For example, we analyzed head shakes expressing negation in Sign Language of the Netherlands (NGT). We discovered that kinematic properties of the head shakes (such as duration, amplitude and velocity) can be affected by linguistic factors, such as whether the headshake spreads across several manual signs, and which signs it spreads over. This showcases our approach: it is possible to quantitatively investigate the formal properties of nonmanuals and how they are affected by linguistic features within and across languages. We are currently working on a larger scale comparison of head shake across several sign languages.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

The current project is among the first studies to showcase the possibility of conducting qunatitative research on nonmanuals using Computer Vision. While the initial results mainly concern singular sign languages and specific nonmanuals, it shows the potential to conduct large scale comparative research on nonmanuals to discover language-specific and universal patterns in the shapes on nonmanuals.

Visualizatio of a head shake in Sign Language of the Netherlands