Fundamentals of formal properties of nonmanuals: A quantitative approach

Información del proyecto

NONMANUAL

Identificador del acuerdo de subvención: 101039378

DOI

10.3030/101039378

Fecha de la firma de la CE 28 Abril 2022

Fecha de inicio 1 Enero 2023

Fecha de finalización 31 Diciembre 2027

Financiado con arreglo a

• European Research Council (ERC)

Coste total

€ 1 500 000,00

Aportación de la UE

€ 1 500 000,00

Coordinado por

UNIVERSITETET I BERGEN
Norway

Periodic Reporting for period 1 - NONMANUAL (Fundamentals of formal properties of nonmanuals: A quantitative approach)

Período documentado: 2023-01-01 hasta 2025-06-30

Resumen del contexto y de los objetivos generales del proyecto

Sign languages, in addition to using the hands, also use positions and movements of other articulators: the body, the head, the mouth, the eyebrows, the eyes and the eyelids, to convey lexical, grammatical, and prosodic information. This linguistic use of the nonmanual articulators is known as nonmanuals. Contrary to current assumptions in the field of sign linguistics, this project proposes the hypothesis that all sign languages use the same basic universal building blocks (nonmanual movements) but that each language is different in how it combines these building blocks both sequentially and simultaneously; languages also differ in the regularity, frequency, and the alignment properties of the nonmanuals. In order to test this hypothesis, we use Computer Vision to measure nonmanuals and advanced statistical techniques to analyze the formal properties of nonmanuals and compare them across contexts and languages. Using these techniques, we analyze naturalistic corpus data (recordings of natural conversations) in several sign languages, and focus on a variety of nonmanuals, such as head shakes used to express negation, head movements marking questions, and eye blinks marking sentence boundaries. We want to find out the extent of variation between the languages in how nonmanuals are used, and the universal patterns undelrying this variation.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

In the first two years of the project, we pursued several research directions.

On the technical side, we have tested and calibrated several Computer Vision solutions, specifically for measuring head movements and detecting eyeblinks. In addition, we have collected a naturalistic data set of Norwegian Sign Language, in which 10 deaf signers produce sentences with various nonmanuals while being recorded by multiple cameras (including depth measuring cameras). We will use this data set for further testing and improvement of Computer Vision instruments. Finally, we have been working on statistical solutions for the analysis of the measurements produced by the instruments. We are developing procedures for dealing with noisy data, detecting the relevant shapes on nonmanuals, and producing various kinematic measurements based on these measurements.

On the linguistic side, we have been quantitatively analyzing various nonmanuals across different sign languages. For example, we analyzed head shakes expressing negation in Sign Language of the Netherlands (NGT). We discovered that kinematic properties of the head shakes (such as duration, amplitude and velocity) can be affected by linguistic factors, such as whether the headshake spreads across several manual signs, and which signs it spreads over. This showcases our approach: it is possible to quantitatively investigate the formal properties of nonmanuals and how they are affected by linguistic features within and across languages. We are currently working on a larger scale comparison of head shake across several sign languages.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

The current project is among the first studies to showcase the possibility of conducting qunatitative research on nonmanuals using Computer Vision. While the initial results mainly concern singular sign languages and specific nonmanuals, it shows the potential to conduct large scale comparative research on nonmanuals to discover language-specific and universal patterns in the shapes on nonmanuals.

Visualizatio of a head shake in Sign Language of the Netherlands