TalkingHeads: Audiovisual Speech Recognition in-the-wild

Projektinformationen

TalkingHeads

ID Finanzhilfevereinbarung: 706668

DOI 10.3030/706668

Projekt abgeschlossen

EK-Unterschriftsdatum 16 März 2016

Startdatum 1 Juni 2016

Enddatum 31 Mai 2018

Finanziert unter

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Gesamtkosten € 183 454,80

EU-Beitrag € 183 454,80

Koordiniert durch

THE UNIVERSITY OF NOTTINGHAM
United Kingdom

Ziel

Audio-visual speech recognition refers to the problem of recognizing speech using both audio and video information. Speech is not a purely auditory process but the way that the listener perceives it is also through the recognition of the visual patterns associated with the mouth movement. This correlation of the audio-visual information has been occasionally explored in literature in order to develop more robust automatic speech recognition systems for cases in which the auditory environment is noisy (e.g. background noise, multiple speakers). However, the problem of audio-visual speech recognition has been mainly studied in controlled, laboratory conditions. TalkingHeads proposes, for the first time, the problem of audio-visual speech recognition in unconstrained (in-the-wild) videos collected from real-world multimedia databases and a set of methodologies that will work well under the assumed in-the-wild setting.

TalkingHeads brings together a talented but experienced researcher (ER) with expertise in speech analysis (diarization and recognition) and the Supervisor with large research experience in Computer Vision for face analysis in-the-wild (recognition, detection, alignment and tracking, and facial expression analysis). TalkingHeads will establish the ER as an independent and internationally recognized researcher in the area of audio-visual fusion and speech recognition. Through TalkingHeads’ achievable work plan, the ER will attain a high level of research maturity by (a) complementing his expertise on speech analysis through extensive training in Computer Vision, (b) conducting research on a challenging research problem (audio-visual speech recognition in-the-wild) with significant career opportunities in both the academia and the industry, (c) publishing at high impact factor conferences and journals, (d) establishing a network of research collaborators, and (e) enhancing personal skills (e.g. supervisory experience, leadership and management skills).

Wissenschaftliches Gebiet

• natural sciences › computer and information sciences › databases
• social sciences › law › law enforcement
• natural sciences › computer and information sciences › artificial intelligence › computer vision
• engineering and technology › electrical engineering, electronic engineering, information engineering › information engineering › telecommunications › mobile phones
• natural sciences › computer and information sciences › artificial intelligence › machine learning › deep learning

Programm/Programme

• H2020-EU.1.3. - EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions Main Programme
• H2020-EU.1.3.2. - Nurturing excellence by means of cross-border and cross-sector mobility

Thema/Themen

• MSCA-IF-2015-EF - Marie Skłodowska-Curie Individual Fellowships (IF-EF)

Aufforderung zur Vorschlagseinreichung

H2020-MSCA-IF-2015

Finanzierungsplan

MSCA-IF-EF-ST - Standard EF

Koordinator

THE UNIVERSITY OF NOTTINGHAM

• Netto-EU-Beitrag: € 183 454,80
• Adresse: University Park ; NG7 2RD Nottingham ; Vereinigtes Königreich
• Region: East Midlands (England) › Derbyshire and Nottinghamshire › Nottingham
• Aktivitätstyp: Higher or Secondary Education Establishments
• Gesamtkosten: € 183 454,80