SignON - Sign Language Translation Mobile Application and Open Communications Framework

Información del proyecto

SignON

Identificador del acuerdo de subvención: 101017255

DOI 10.3030/101017255

Proyecto cerrado

Fecha de la firma de la CE 30 Noviembre 2020

Fecha de inicio 1 Enero 2021

Fecha de finalización 31 Diciembre 2023

Financiado con arreglo a

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Coste total € 5 616 718,75

Aportación de la UE € 5 616 718,75

Coordinado por

DUBLIN CITY UNIVERSITY
Ireland

Resultado final

Other (12)

First Sign language-specific lexicon and structure
Second Sign language-specific lexicon and structure
Sign language recognition component and models

ign language recognition component and models

Final Sign language-specific lexicon and structure
Automatic speech recognition component and models
SignON platform and cloud infrastructure

Definition of the platform and cloud infrastructure for the project

First symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

First symbolic intermediate representation - InterL-S

Linguistic description for ISL, BSL, VGT, NGT and LSE
First distributional intermediate representation based on embeddings - InterL-E
Second distributional intermediate representation based on embeddings - InterL-E
A hybrid intermediate representation

Documents, reports (26)

First Annual Ethical Report on Practices Towards the Stakeholder Communities

D921 Annual Ethical Report on Practices Towards the Stakeholder Communities

SignON Market Analysis
Second Annual Ethical Report on Internal Affairs

D9.3.2. Annual Ethical Report on Internal Affairs

Final Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.3 Annual Ethical Report on Practices Towards the Stakeholder Communities

First Annual reports on communication and dissemination activities
Final Annual reports on communication and dissemination activities
SignON Communication and Dissemination Plan
Quality Assurance plan
Report on the use-cases and usage domains and stakeholders' acceptance.

Report on use cases and acceptance

Case studies and evidence analysis

Description of case studies

Interim progress report

First Annual progress report

Report on Ethical approval

Report on the ethical approvals prepared, signed and collected during the project

First Interim Community building Report.

Report on community building

Second Impact Review
Final Annual Ethical Report on Internal Affairs

D9.3.3. Annual Ethical Report on Internal Affairs

Quality Assessment Report

Report on Quality of the sign on service

Ethical Guidelines and Protocols

D81 Ethical Guidelines and Protocols including an overview of how all data that will be processed is relevant and limited to the purposes of the SignOn project to ensure that we operate in accordance with the data minimisation principle M6

First Annual Ethical Report on Internal Affairs

D931 Annual Ethical Report on Internal Affairs

First Impact Review
Second Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.2. Annual Ethical Report on Practices Towards the Stakeholder Communities

Final Impact Review
Final Report
Data Transfer Agreement

Agreement between all partners on the transference of data

Second Annual reports on communication and dissemination activities
Workshops, showcases, demonstrations report
Risk Management Plan

Demonstrators, pilots, prototypes (10)

Final release of the Communication Mobile Application

Final release of the Communication Mobile Application (demonstrator/code)

Second Natural Language processing pipeline
First release of the Open SignON Framework

First release of the framework (report + demonstrator)

First Natural Language Processing Pipelines
Final release of the Open SignON Framework

Final release of the Open SignON Framework (report + demonstrator)

Final version of virtual character
Intermediate release of the Open SignON Framework

Intermediate release of the Open SignON Framework (Report + demonstrator)

First release of the SignON Communication Mobile

First release of the SignON Communication Mobile (demonstrator/code)

First version of virtual character
Interactive co-creation web-based platform for learning from user input

Open Research Data Pilot (2)

Data Management Report

Report on Data management for the duration of the action

Data Management Plan

including a description of the technical and organisational measures that partners must implement to safeguard the rights and freedoms of research participants and the security measures that will be implemented to prevent unauthorised access to personal data or the equipment used for processing data

Publicaciones

Conference proceedings (11)

Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention

Autores: Mathieu De Coster, Mieke Van Herreweghe, Joni Dambre
Publicado en: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Editor: IEEE
DOI: 10.1109/cvprw53098.2021.00383

SignON: Bridging the gap between Sign and Spoken Languages

Autores: Horacio Saggion, Dimitar Shterionov, Gorka Labaka, Tim Van de Cruys, Vincent Vandeghinste and Josep Blat
Publicado en: Proceedings of the Annual Conference of the Spanish Association for Natural Language Processing: Projects and Demonstrations (SEPLN-PD 2021) co-located with the Conference of the Spanish Society for Natural Language Processing (SEPLN 2021), Edición Annual, 2022, Página(s) 21-25
Editor: CEUR Workshop Proceedings

Challenges with Sign Language Datasets for Sign Language Recognition and Translation.

Autores: Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov and Horacio Saggion
Publicado en: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC), 2022
Editor: European Language Resources Association

Frozen Pretrained Transformers for Neural Sign Language Translation.

Autores: Mathieu De Coster, Karel D’Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, Joni Dambre.
Publicado en: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021
Editor: MT Summit

Sign Language Fingerspelling Recognition using Synthetic Data

Autores: Frank Fowley and Anthony Ventresque
Publicado en: Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), 2021
Editor: AICS

Language Planning in Action: Depiction as a Driver of New Terminology in Irish Sign Language.

Autores: Rachel Moiselle and Lorraine Leeson
Publicado en: Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022), 2022, Página(s) 139-143
Editor: European Language Resources Association

Sign Language Translation: Ongoing Development, Challenges and Innovations in the SignON Project

Autores: Dimitar Shterionov, Mirella De Sisto, Vincent Vandeghinste, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert
Publicado en: Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022
Editor: European Association for Machine Translation

Improving Signer Independent Sign Language Recognition for Low Resource Languages. Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), pp. 45-52. Marseille, France.

Autores: Ruth Holmes, Ellen Rushe, Frank Fowley, Anthony Ventresque
Publicado en: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), 2022, Página(s) 45-52
Editor: European Language Resources Association

Syntax-aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation.

Autores: Santiago Egea Gómez, Euan McGill, Horacio Saggion
Publicado en: Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC2021), 2021, Página(s) 18-27
Editor: INCOMA Ltd.

Defining Meaningful Units: Challenges in Sign Segmentation and Segment-Meaning Mapping

Autores: Mirella De Sisto, Dimitar Shterionov, Irene Murtagh, Myriam Vermeerbergen and Lorraine Leeson
Publicado en: 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL 2021), 2021, Página(s) 98-103
Editor: online

Attention Analysis of a Sign Language Recognition Task on the AUTSL Dataset

Autores: Jeanne Coppin, Mathieu De Coster, Joni Dambre
Publicado en: 32nd Meeting of Computational Linguistics in The Netherlands (CLIN32), Edición Annual, 2022
Editor: Tilburg University

Other (2)

Design principles of an Automatic Speech Recognition functionality in a user-centric signed and spoken language translation system

Autores: Aditya Parikh, Louis ten Bosch, Henk van den Heuvel, Cristian Tejedor-García
Publicado en: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Editor: Tilburg University

Progress, Challenges and Innovations of the SignON Project

Autores: Vincent Vandeghinste, Mirella De Sisto, Dimitar Shterionov, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Scipioni, Aditya Parikh, Louis ten Bosch, John O'Flaherty, Joni Dambre and Jorn Rijckaert
Publicado en: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Editor: Tilburg University

Peer reviewed articles (1)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation

Autores: Mathieu De Coster, Joni Dambre
Publicado en: Information, 2022, ISSN 2078-2489
Editor: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13050220