SignON - Sign Language Translation Mobile Application and Open Communications Framework

Informacje na temat projektu

SignON

Identyfikator umowy o grant: 101017255

DOI 10.3030/101017255

Projekt został zamknięty

Data podpisania przez KE 30 Listopada 2020

Data rozpoczęcia 1 Stycznia 2021

Data zakończenia 31 Grudnia 2023

Finansowanie w ramach

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Koszt całkowity € 5 616 718,75

Wkład UE € 5 616 718,75

Koordynowany przez

DUBLIN CITY UNIVERSITY
Ireland

Rezultaty

Other (12)

First Sign language-specific lexicon and structure
Second Sign language-specific lexicon and structure
Sign language recognition component and models

ign language recognition component and models

Final Sign language-specific lexicon and structure
Automatic speech recognition component and models
SignON platform and cloud infrastructure

Definition of the platform and cloud infrastructure for the project

First symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

First symbolic intermediate representation - InterL-S

Linguistic description for ISL, BSL, VGT, NGT and LSE
First distributional intermediate representation based on embeddings - InterL-E
Second distributional intermediate representation based on embeddings - InterL-E
A hybrid intermediate representation

Documents, reports (26)

First Annual Ethical Report on Practices Towards the Stakeholder Communities

D921 Annual Ethical Report on Practices Towards the Stakeholder Communities

SignON Market Analysis
Second Annual Ethical Report on Internal Affairs

D9.3.2. Annual Ethical Report on Internal Affairs

Final Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.3 Annual Ethical Report on Practices Towards the Stakeholder Communities

First Annual reports on communication and dissemination activities
Final Annual reports on communication and dissemination activities
SignON Communication and Dissemination Plan
Quality Assurance plan
Report on the use-cases and usage domains and stakeholders' acceptance.

Report on use cases and acceptance

Case studies and evidence analysis

Description of case studies

Interim progress report

First Annual progress report

Report on Ethical approval

Report on the ethical approvals prepared, signed and collected during the project

First Interim Community building Report.

Report on community building

Second Impact Review
Final Annual Ethical Report on Internal Affairs

D9.3.3. Annual Ethical Report on Internal Affairs

Quality Assessment Report

Report on Quality of the sign on service

Ethical Guidelines and Protocols

D81 Ethical Guidelines and Protocols including an overview of how all data that will be processed is relevant and limited to the purposes of the SignOn project to ensure that we operate in accordance with the data minimisation principle M6

First Annual Ethical Report on Internal Affairs

D931 Annual Ethical Report on Internal Affairs

First Impact Review
Second Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.2. Annual Ethical Report on Practices Towards the Stakeholder Communities

Final Impact Review
Final Report
Data Transfer Agreement

Agreement between all partners on the transference of data

Second Annual reports on communication and dissemination activities
Workshops, showcases, demonstrations report
Risk Management Plan

Demonstrators, pilots, prototypes (10)

Final release of the Communication Mobile Application

Final release of the Communication Mobile Application (demonstrator/code)

Second Natural Language processing pipeline
First release of the Open SignON Framework

First release of the framework (report + demonstrator)

First Natural Language Processing Pipelines
Final release of the Open SignON Framework

Final release of the Open SignON Framework (report + demonstrator)

Final version of virtual character
Intermediate release of the Open SignON Framework

Intermediate release of the Open SignON Framework (Report + demonstrator)

First release of the SignON Communication Mobile

First release of the SignON Communication Mobile (demonstrator/code)

First version of virtual character
Interactive co-creation web-based platform for learning from user input

Open Research Data Pilot (2)

Data Management Report

Report on Data management for the duration of the action

Data Management Plan

including a description of the technical and organisational measures that partners must implement to safeguard the rights and freedoms of research participants and the security measures that will be implemented to prevent unauthorised access to personal data or the equipment used for processing data

Publikacje

Conference proceedings (11)

Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention

Autorzy: Mathieu De Coster, Mieke Van Herreweghe, Joni Dambre
Opublikowane w: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Wydawca: IEEE
DOI: 10.1109/cvprw53098.2021.00383

SignON: Bridging the gap between Sign and Spoken Languages

Autorzy: Horacio Saggion, Dimitar Shterionov, Gorka Labaka, Tim Van de Cruys, Vincent Vandeghinste and Josep Blat
Opublikowane w: Proceedings of the Annual Conference of the Spanish Association for Natural Language Processing: Projects and Demonstrations (SEPLN-PD 2021) co-located with the Conference of the Spanish Society for Natural Language Processing (SEPLN 2021), Numer Annual, 2022, Strona(/y) 21-25
Wydawca: CEUR Workshop Proceedings

Challenges with Sign Language Datasets for Sign Language Recognition and Translation.

Autorzy: Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov and Horacio Saggion
Opublikowane w: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC), 2022
Wydawca: European Language Resources Association

Frozen Pretrained Transformers for Neural Sign Language Translation.

Autorzy: Mathieu De Coster, Karel D’Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, Joni Dambre.
Opublikowane w: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021
Wydawca: MT Summit

Sign Language Fingerspelling Recognition using Synthetic Data

Autorzy: Frank Fowley and Anthony Ventresque
Opublikowane w: Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), 2021
Wydawca: AICS

Language Planning in Action: Depiction as a Driver of New Terminology in Irish Sign Language.

Autorzy: Rachel Moiselle and Lorraine Leeson
Opublikowane w: Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022), 2022, Strona(/y) 139-143
Wydawca: European Language Resources Association

Sign Language Translation: Ongoing Development, Challenges and Innovations in the SignON Project

Autorzy: Dimitar Shterionov, Mirella De Sisto, Vincent Vandeghinste, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert
Opublikowane w: Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022
Wydawca: European Association for Machine Translation

Improving Signer Independent Sign Language Recognition for Low Resource Languages. Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), pp. 45-52. Marseille, France.

Autorzy: Ruth Holmes, Ellen Rushe, Frank Fowley, Anthony Ventresque
Opublikowane w: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), 2022, Strona(/y) 45-52
Wydawca: European Language Resources Association

Syntax-aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation.

Autorzy: Santiago Egea Gómez, Euan McGill, Horacio Saggion
Opublikowane w: Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC2021), 2021, Strona(/y) 18-27
Wydawca: INCOMA Ltd.

Defining Meaningful Units: Challenges in Sign Segmentation and Segment-Meaning Mapping

Autorzy: Mirella De Sisto, Dimitar Shterionov, Irene Murtagh, Myriam Vermeerbergen and Lorraine Leeson
Opublikowane w: 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL 2021), 2021, Strona(/y) 98-103
Wydawca: online

Attention Analysis of a Sign Language Recognition Task on the AUTSL Dataset

Autorzy: Jeanne Coppin, Mathieu De Coster, Joni Dambre
Opublikowane w: 32nd Meeting of Computational Linguistics in The Netherlands (CLIN32), Numer Annual, 2022
Wydawca: Tilburg University

Other (2)

Design principles of an Automatic Speech Recognition functionality in a user-centric signed and spoken language translation system

Autorzy: Aditya Parikh, Louis ten Bosch, Henk van den Heuvel, Cristian Tejedor-García
Opublikowane w: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Wydawca: Tilburg University

Progress, Challenges and Innovations of the SignON Project

Autorzy: Vincent Vandeghinste, Mirella De Sisto, Dimitar Shterionov, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Scipioni, Aditya Parikh, Louis ten Bosch, John O'Flaherty, Joni Dambre and Jorn Rijckaert
Opublikowane w: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Wydawca: Tilburg University

Peer reviewed articles (1)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation

Autorzy: Mathieu De Coster, Joni Dambre
Opublikowane w: Information, 2022, ISSN 2078-2489
Wydawca: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13050220