CORDIS
EU research results

CORDIS

English EN

Ecological Language: A multimodal approach to language and the brain

Objective

The human brain has evolved the ability to support communication in complex and dynamic environments. In such environments, language is learned, and mostly used in face-to-face contexts in which processing and learning is based on multiple cues: linguistic (such as lexical, syntactic), but also discourse, prosody, face and hands (gestures). Yet, our understanding of how language is learnt and processed, and its associated neural circuitry, comes almost exclusively from reductionist approaches in which the multimodal signal is reduced to speech or text. ECOLANG will pioneer a new way to study language comprehension and learning using a real-world approach in which language is analysed in its rich face-to-face multimodal environment (i.e., language’s ecological niche). Experimental rigour is not compromised by the use of innovative technologies (combining automatic, manual and crowdsourcing methods for annotation; creating avatar stimuli for our experiments) and state-of-the-art modelling and data analysis (probabilistic modelling and network-based analyses). ECOLANG studies how the different cues available in face-to-face communication dynamically contribute to processing and learning in adults, children and aphasic patients in contexts representative of everyday conversation. We collect and annotate a corpus of naturalistic language which is then used to derive quantitative informativeness measures for each cue and their combination using computational models, tested and refined on the basis of behavioural and neuroscientific data. We use converging methodologies (behavioural, EEG, fMRI and lesion-symptom mapping) and we investigate different populations (3-4 years old children, healthy and aphasic adults) in order to develop mechanistic accounts of multimodal communication at the cognitive as well as neural level that can explain processing and learning (by both children and adults) and can have impact on the rehabilitation of language functions after stroke.

Host institution

UNIVERSITY COLLEGE LONDON

Address

Gower Street
Wc1e 6bt London

United Kingdom

Activity type

Higher or Secondary Education Establishments

EU Contribution

€ 2 221 020

Beneficiaries (3)

Sort alphabetically

Sort by EU Contribution

Expand all

UNIVERSITY COLLEGE LONDON

United Kingdom

EU Contribution

€ 2 221 020

UNIVERSITY OF BRIGHTON

United Kingdom

EU Contribution

€ 12 414,24

UNIVERSITAET ZU KOELN

Germany

EU Contribution

€ 10 149,76

Project information

Grant agreement ID: 743035

Status

Ongoing project

  • Start date

    1 January 2018

  • End date

    31 December 2022

Funded under:

H2020-EU.1.1.

  • Overall budget:

    € 2 243 584

  • EU contribution

    € 2 243 584

Hosted by:

UNIVERSITY COLLEGE LONDON

United Kingdom