Skip to main content

SignON - Sign Language Translation Mobile Application and Open Communications Framework

Objective

Nowadays any communication barrier is detrimental to society. This project will research and develop the SignON communication service that uses machine translation to translate between Sign and spoken languages. This service will facilitate the exchange of information among deaf and hard of hearing, and hearing individuals. In this user-centric and community-driven project we will tightly collaborate with European deaf and hard of hearing communities to (re)define use-cases, co-design and co-develop the SignON service and application, assess the quality and validate their acceptance. Our ultimate objective is the fair, unbiased and inclusive spread of information and digital content in European society.
Our project will develop a free, open application and framework for conversion between video (capturing and understanding Sign language), audio and text and translation between Sign and spoken languages. To facilitate these tasks we propose a common representation for mapping of video, audio and text into a unified space, that will be used for translating into the target modality and language. To ensure wide uptake, improved sign language detection and synthesis, as well as multilingual speech processing on mobile devices for everyone, we will deploy the SignON service as a smart phone application running on standard modern devices. We envisage the application as a light-weight interface. The SignON framework, however, will be distributed on the cloud where the computationally intensive tasks will be executed.The project will be driven by a focused set of use-cases tailored towards the deaf communities. We target the Irish, British, Dutch, Flemish and Spanish Sign and English, Irish, Dutch, Spanish spoken languages. However, SignON will incorporate sophisticated machine learning capabilities that will allow (i) learning new Sign, written and spoken languages; (ii) style-, domain- and user-adaptation and (iii) automatic error correction, based on user feedback

Field of science

  • /humanities/languages and literature/linguistics/sign language
  • /humanities/arts/modern and contemporary art/film

Call for proposal

H2020-ICT-2020-2
See other projects for this call

Funding Scheme

RIA - Research and Innovation action

Coordinator

DUBLIN CITY UNIVERSITY
Address
Glasnevin
9 Dublin
Ireland
Activity type
Higher or Secondary Education Establishments
EU contribution
€ 430 755

Participants (16)

FINCONS GROUP AG
Switzerland
EU contribution
€ 442 500
Address
Bahnhofstrasse 57
6403 Kussnacht
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
STICHTING INSTITUUT VOOR DE NEDERLANDSE TAAL
Netherlands
EU contribution
€ 263 375
Address
Rapenburg 61
2311 GJ Leiden
Activity type
Research Organisations
UNIVERSIDAD DEL PAIS VASCO/ EUSKAL HERRIKO UNIBERTSITATEA
Spain
EU contribution
€ 231 250
Address
Barrio Sarriena S N
48940 Leioa
Activity type
Higher or Secondary Education Establishments
THE NATIONAL MICROELECTRONICS APPLICATIONS CENTRE LTD
Ireland
EU contribution
€ 342 500
Address
Suparule House, Lonsdale Road National Technologicy Park
. Limerick
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
UNIVERSIDAD POMPEU FABRA
Spain
EU contribution
€ 539 375
Address
Placa De La Merce, 10-12
08002 Barcelona
Activity type
Higher or Secondary Education Establishments
TECHNOLOGICAL UNIVERSITY DUBLIN
Ireland
EU contribution
€ 269 758,75
Address
North Circular Road 191 Park House Grangegorman
D07 EWV4 Dublin
Activity type
Higher or Secondary Education Establishments
THE PROVOST, FELLOWS, FOUNDATION SCHOLARS & THE OTHER MEMBERS OF BOARD OF THE COLLEGE OF THE HOLY & UNDIVIDED TRINITY OF QUEEN ELIZABETH NEAR DUBLIN
Ireland
EU contribution
€ 242 236,25
Address
College Green
2 Dublin
Activity type
Higher or Secondary Education Establishments
DE VLAAMSE RADIO EN TELEVISIEOMROEPORGANISATIE NV
Belgium
EU contribution
€ 288 000
Address
Auguste Reyerslaan 52
1043 Brussel
Activity type
Other
UNIVERSITEIT GENT
Belgium
EU contribution
€ 331 295
Address
Sint Pietersnieuwstraat 25
9000 Gent
Activity type
Higher or Secondary Education Establishments
VLAAMS GEBARENTAALCENTRUM VZW
Belgium
EU contribution
€ 364 250
Address
Sint-andriesstraat 2
2000 Antwerpen
Activity type
Research Organisations
UNIVERSITY COLLEGE DUBLIN, NATIONAL UNIVERSITY OF IRELAND, DUBLIN
Ireland
EU contribution
€ 207 271,25
Address
Belfield
4 Dublin
Activity type
Higher or Secondary Education Establishments
STICHTING KATHOLIEKE UNIVERSITEIT
Netherlands
EU contribution
€ 424 500
Address
Geert Grooteplein Noord 9
6525 EZ Nijmegen
Activity type
Higher or Secondary Education Establishments
NEDERLANDSE TAALUNIE
Netherlands
EU contribution
€ 94 500
Address
Paleisstraat 9
2514JA Den Haag
Activity type
Public bodies (excluding Research Organisations and Secondary or Higher Education Establishments)
KATHOLIEKE UNIVERSITEIT LEUVEN
Belgium
EU contribution
€ 492 800
Address
Oude Markt 13
3000 Leuven
Activity type
Higher or Secondary Education Establishments
EUROPEAN UNION OF THE DEAF AISBL
Belgium
EU contribution
€ 373 500
Address
Wetstraat 26 15
1040 Brussel
Activity type
Research Organisations
STICHTING KATHOLIEKE UNIVERSITEIT BRABANT
Netherlands
EU contribution
€ 278 852,50
Address
Warandelaan 2
5037 AB Tilburg
Activity type
Higher or Secondary Education Establishments