SignON - Sign Language Translation Mobile Application and Open Communications Framework

Información del proyecto

SignON

Identificador del acuerdo de subvención: 101017255

DOI

10.3030/101017255

Proyecto cerrado

Fecha de la firma de la CE 30 Noviembre 2020

Fecha de inicio 1 Enero 2021

Fecha de finalización 31 Diciembre 2023

Financiado con arreglo a

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Coste total

€ 5 616 718,75

Aportación de la UE

€ 5 616 718,75

Coordinado por

DUBLIN CITY UNIVERSITY
Ireland

CORDIS proporciona enlaces a los documentos públicos y las publicaciones de los proyectos de los programas marco HORIZONTE.

Los enlaces a los documentos y las publicaciones de los proyectos del Séptimo Programa Marco, así como los enlaces a algunos tipos de resultados específicos, como conjuntos de datos y «software», se obtienen dinámicamente de OpenAIRE .

Resultado final

Other (12)

First Sign language-specific lexicon and structure
Second Sign language-specific lexicon and structure
Sign language recognition component and models

ign language recognition component and models

Final Sign language-specific lexicon and structure
Automatic speech recognition component and models
SignON platform and cloud infrastructure

Definition of the platform and cloud infrastructure for the project

First symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

First symbolic intermediate representation - InterL-S

Linguistic description for ISL, BSL, VGT, NGT and LSE
First distributional intermediate representation based on embeddings - InterL-E
Second distributional intermediate representation based on embeddings - InterL-E
A hybrid intermediate representation

Documents, reports (26)

First Annual Ethical Report on Practices Towards the Stakeholder Communities

D921 Annual Ethical Report on Practices Towards the Stakeholder Communities

SignON Market Analysis
Second Annual Ethical Report on Internal Affairs

D9.3.2. Annual Ethical Report on Internal Affairs

Final Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.3 Annual Ethical Report on Practices Towards the Stakeholder Communities

First Annual reports on communication and dissemination activities
Final Annual reports on communication and dissemination activities
SignON Communication and Dissemination Plan
Quality Assurance plan
Report on the use-cases and usage domains and stakeholders' acceptance.

Report on use cases and acceptance

Case studies and evidence analysis

Description of case studies

Interim progress report

First Annual progress report

Report on Ethical approval

Report on the ethical approvals prepared, signed and collected during the project

First Interim Community building Report.

Report on community building

Second Impact Review
Final Annual Ethical Report on Internal Affairs

D9.3.3. Annual Ethical Report on Internal Affairs

Quality Assessment Report

Report on Quality of the sign on service

Ethical Guidelines and Protocols

D81 Ethical Guidelines and Protocols including an overview of how all data that will be processed is relevant and limited to the purposes of the SignOn project to ensure that we operate in accordance with the data minimisation principle M6

First Annual Ethical Report on Internal Affairs

D931 Annual Ethical Report on Internal Affairs

First Impact Review
Second Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.2. Annual Ethical Report on Practices Towards the Stakeholder Communities

Final Impact Review
Final Report
Data Transfer Agreement

Agreement between all partners on the transference of data

Second Annual reports on communication and dissemination activities
Workshops, showcases, demonstrations report
Risk Management Plan

Demonstrators, pilots, prototypes (10)

Final release of the Communication Mobile Application

Final release of the Communication Mobile Application (demonstrator/code)

Second Natural Language processing pipeline
First release of the Open SignON Framework

First release of the framework (report + demonstrator)

First Natural Language Processing Pipelines
Final release of the Open SignON Framework

Final release of the Open SignON Framework (report + demonstrator)

Final version of virtual character
Intermediate release of the Open SignON Framework

Intermediate release of the Open SignON Framework (Report + demonstrator)

First release of the SignON Communication Mobile

First release of the SignON Communication Mobile (demonstrator/code)

First version of virtual character
Interactive co-creation web-based platform for learning from user input

Open Research Data Pilot (2)

Data Management Report

Report on Data management for the duration of the action

Data Management Plan

including a description of the technical and organisational measures that partners must implement to safeguard the rights and freedoms of research participants and the security measures that will be implemented to prevent unauthorised access to personal data or the equipment used for processing data

Publicaciones

Other (7)

Onderzoek naar constituentenvolgorde in Vlaamse Gebarentaal. Enkelvoudige mededelende zinnen bij jongeren van 12 tot 18 jaar

Autores: Lien Soetemans and Margot Janssens
Publicado en: Onderzoeksrapport KU Leuven, 2023
Editor: N/A

Realising Inclusive Design Through Co-creation

Autores: Brady, Aoife, Clarke, Emma, Dunne, Jane, Kearns, Anne, Leeson, Lorraine, Mathews, Elizabeth, Moiselle, Rachel, O’Boyle, Shaun, Omardeen, Rehana, Picron, Frankie, Shterionov, Dimitar, Van Landuyt, Davy
Publicado en: EARMA Conference, 2023
Editor: EARMA

Design principles of an Automatic Speech Recognition functionality in a user-centric signed and spoken language translation system

Autores: Aditya Parikh, Louis ten Bosch, Henk van den Heuvel, Cristian Tejedor-García
Publicado en: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Editor: Tilburg University

Using co-creation to develop sign language machine translation technologies (Abstract)

Autores: Shaun O’Boyle, Elizabeth Mathews, Davy Van Landuyt, Frankie Picron, Rehana Omardeen, Lorraine Leeson, Rachel Moiselle, Aoife Brady, Jorn Rijckaert, Caro Brosens, Anthony Ventresque, Ellen Rushe, Irene Murtagh, Andy Way, Dimitar Shterionov
Publicado en: Poster presentation at SCI:COM 2022, Dublin, Ireland, 2022
Editor: N/A

Voorgetrainde Grote Taalmodellen en Automatische Vertaling.

Autores: Vincent Vandeghinste and Tim Van de Cruys
Publicado en: Dixit. Tijdschrift voor Taaltechnologie, 2022
Editor: Dixit

Progress, Challenges and Innovations of the SignON Project

Autores: Vincent Vandeghinste, Mirella De Sisto, Dimitar Shterionov, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Scipioni, Aditya Parikh, Louis ten Bosch, John O'Flaherty, Joni Dambre and Jorn Rijckaert
Publicado en: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Editor: Tilburg University

WordNets? SignNets!

Autores: Ineke Schuurman, Vincent Vandeghinste, Caro Brosens, Margot Janssens and Thierry Declerck
Publicado en: Presented at the 32nd Meeting of Computational Linguistics in The Netherlands (CLIN 32), 2022
Editor: CLIN

Conference proceedings (25)

A Data Augmentation and Pre-processing Technique for Sign Language Fingerspelling Recognition

Autores: Frank Fowley, Ellen Rushe and Anthony Ventresque
Publicado en: 24th Irish Machine Vision and Image Processing Conference, 2022
Editor: IMVIPC
DOI: 10.56541/xbav3102

BSL-Hansard: A parallel, multimodal corpus of English and interpreted British Sign Language data from parliamentary proceedings

Autores: Euan McGill, Horacio Saggion.
Publicado en: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Editor: European Association for Machine Translation

Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention

Autores: Mathieu De Coster, Mieke Van Herreweghe, Joni Dambre
Publicado en: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Editor: IEEE
DOI: 10.1109/cvprw53098.2021.00383

SignON: Bridging the gap between Sign and Spoken Languages

Autores: Horacio Saggion, Dimitar Shterionov, Gorka Labaka, Tim Van de Cruys, Vincent Vandeghinste and Josep Blat
Publicado en: Proceedings of the Annual Conference of the Spanish Association for Natural Language Processing: Projects and Demonstrations (SEPLN-PD 2021) co-located with the Conference of the Spanish Society for Natural Language Processing (SEPLN 2021), Edición Annual, 2022, Página(s) 21-25
Editor: CEUR Workshop Proceedings

SignON Sign Language Translation : Progress and Challenges

Autores: Vincent Vandeghinste, Dimitar Shterionov, Mirella De Sisto, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert, Bram Vanroy, Victor Ubieto Nogales, Santiago Egea Gomez, Ineke Schuurman, Gorka Labaka, Adrian Nunez-Marcos, Irene Murtagh, Euan McGill and Horacio Saggion
Publicado en: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023, ISBN 9789520329471
Editor: European Association for Machine Translation (EAMT)

Challenges with Sign Language Datasets for Sign Language Recognition and Translation.

Autores: Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov and Horacio Saggion
Publicado en: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC), 2022
Editor: European Language Resources Association

From Scarcity to Understanding: Transfer Learning for the Extremely Low Resource Irish Sign Language

Autores: Ruth Holmes, Ellen Rushe, Mathieu De Coster, Maxim Bonnaerens, Shin'ichi Satoh, Akihiro Sugimoto, and Anthony Ventresque
Publicado en: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023
Editor: IEEE
DOI: 10.1109/iccvw60793.2023.00215

GoSt-ParC-Sign: Gold Standard Parallel Corpus of Sign and spoken language.

Autores: Mirella De Sisto, Vincent Vandeghinste, Lien Soetemans, Caro Brosens, Dimitar Shterionov
Publicado en: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023
Editor: European Association for Machine Translation

Comparing Modular and End-To-End Approaches in ASR for Well-Resourced and Low-Resourced Languages

Autores: Parikh, A, Ten Bosch, L., Van den Heuvel, H. & C. Tejedor Garcia
Publicado en: In: Proceedings of The 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), 2023
Editor: Association for Computational Linguistics

Part-of-Speech tagging Spanish Sign Language data and its applications in Sign Language machine translation

Autores: Euan McGill, Luis Chiruzzo, Santiago Egea-Gómez, Horacio Saggion
Publicado en: Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains, 2023
Editor: ACL

Frozen Pretrained Transformers for Neural Sign Language Translation.

Autores: Mathieu De Coster, Karel D’Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, Joni Dambre.
Publicado en: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021
Editor: MT Summit

A Linked Data Approach for linking and aligning Sign Language and Spoken Language Data.

Autores: Thierry Declerck, Sam Bigeard, Fahad Khan, Irene Murtagh, Sussi Olsen, Mike Rosner, Ineke Schuurman, Andon Tchechmedjiev, and Andy Way
Publicado en: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Editor: European Association for Machine Translation

Sign Language Machine Translation and the Sign Language Lexicon: A Linguistically Informed Approach

Autores: Irene Murtagh, Víctor Ubieto Nogales, Josep Blat
Publicado en: In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), 2022
Editor: Association for Machine Translation in the Americas

Sign Language Fingerspelling Recognition using Synthetic Data

Autores: Frank Fowley and Anthony Ventresque
Publicado en: Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), 2021
Editor: AICS

Language Planning in Action: Depiction as a Driver of New Terminology in Irish Sign Language.

Autores: Rachel Moiselle and Lorraine Leeson
Publicado en: Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022), 2022, Página(s) 139-143
Editor: European Language Resources Association

Sign Language Translation: Ongoing Development, Challenges and Innovations in the SignON Project

Autores: Dimitar Shterionov, Mirella De Sisto, Vincent Vandeghinste, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert
Publicado en: Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022
Editor: European Association for Machine Translation

Trends and challenges for sign language recognition with machine learning

Autores: Jérôme Fink, Mathieu De Coster, Joni Dambre, Benoît Frénay
Publicado en: Proceedings of the 2023 European Symposium on Artificial Neural Networks, 2023, ISBN 9782875870889
Editor: ESANN
DOI: 10.14428/esann/2023.es2023-7

Improving Signer Independent Sign Language Recognition for Low Resource Languages. Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), pp. 45-52. Marseille, France.

Autores: Ruth Holmes, Ellen Rushe, Frank Fowley, Anthony Ventresque
Publicado en: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), 2022, Página(s) 45-52
Editor: European Language Resources Association

Syntax-aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation.

Autores: Santiago Egea Gómez, Euan McGill, Horacio Saggion
Publicado en: Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC2021), 2021, Página(s) 18-27
Editor: INCOMA Ltd.

NGT-HoReCo and GoSt-ParC-Sign: Two new Sign Language - Spoken Language parallel corpora

Autores: Mirella De Sisto, Dimitar Shterionov, Lien Soetemans, Vincent Vandeghinste, Caro Brosens
Publicado en: Proceedings of the CLARIN Annual Conference; 2023, 2023
Editor: Krister Lindén, Jyrki Niemi, and Thalassia Kontino

Defining Meaningful Units: Challenges in Sign Segmentation and Segment-Meaning Mapping

Autores: Mirella De Sisto, Dimitar Shterionov, Irene Murtagh, Myriam Vermeerbergen and Lorraine Leeson
Publicado en: 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL 2021), 2021, Página(s) 98-103
Editor: online

Translating Spanish into Spanish Sign Language: Combining Rules and Data-driven Approaches

Autores: Luis Chiruzzo, Euan McGill, Santiago Egea-Gómez, Horacio Saggion
Publicado en: Proceedings of the 29th International Conference on Computational Linguistics, 2022
Editor: International Committee on Computational Linguistics

Querying A Sign Language Dictionary with Videos Using Dense Vector Search

Autores: Mathieu De Coster and Joni Dambre
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Editor: IEEE

When shared space and time don’t matter. Remote video-mediated (synchronous and asynchronous ) communication in Flemish Sign Language.

Autores: Lien Soetemans and Myriam Vermeerbergen
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Editor: IEEE
DOI: 10.1109/icasspw59220.2023.10193316

Attention Analysis of a Sign Language Recognition Task on the AUTSL Dataset

Autores: Jeanne Coppin, Mathieu De Coster, Joni Dambre
Publicado en: 32nd Meeting of Computational Linguistics in The Netherlands (CLIN32), Edición Annual, 2022
Editor: Tilburg University

Peer reviewed articles (1)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation

Autores: Mathieu De Coster, Joni Dambre
Publicado en: Information, 2022, ISSN 2078-2489
Editor: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13050220

Book chapters (1)

Linguistically Enhanced Text to Sign Gloss Machine Translation

Autores: Egea Gómez, S., Chiruzzo, L., McGill, E., Saggion, H.
Publicado en: Lecture Notes in Computer Science, vol 13286, 2022
Editor: Springer
DOI: 10.1007/978-3-031-08473-7_16