Intelligent Automatic Sign Language Translation

Resultado final

Report on standardization issues regarding avatar SL representation and the Signing Avatar Challenge Dataset. This deliverable is produced by Task 2.3.

Overview of types of text resources in broadcast data

This report from T63 details multichannel data format used in broadcast data in the light of making these data available for processing within EASIER

Report on interim evaluation study

Report on the interim user evaluation study. This deliverable is produced by Task T1.3.

Dissemination, communication and community engagement report 1

This document will provide an overview of the overall activities carried out gathering input from Tasks 101 and 102 By closely monitoring KPIs and achieved performance across all activities it will be possible to inject back into planned work and adapt the strategy and plans see T101 This deliverable is produced by Task 101 and gets input from Task 102

Specifications for liaising EASIER with ELG

Report on the extension and upgrade of the ELG infrastructure to accommodate EASIER results. This deliverable is produced by Task 8.6.

Statistical data from video processing other European SL datasets

Reports on the results of T9.4 working with the infrastructure set up in D9.6.

Specification for the harmonization of SL annotations

This report produced by T62 suggests an common interchange format for annotations as used in the datasets identified in D61 and provides a reference implementation for converters to this format

Dissemination, communication and community engagement report 2

This updated document will provide an overview of the overall activities carried out, gathering input from Tasks 10.1 and 10.2. By closely monitoring KPIs and achieved performance across all activities, it will be possible to inject back into planned work and adapt the strategy and plans (see T10.1). This deliverable is produced by Task 10.1 and gets input from Task 10.2.

Workflow documents and manuals for the creation of new resources

Is teaching material created in T9.3 supplemented by a helpdesk. It will be used in D9.4.

Impact creation strategy and plan

This document will define the communication and dissemination strategy and describe the activities EASIER will pursue to guarantee broad visibility promotion and uptake of the projectdriven activities This deliverable is produced by Task 101

Report on final evaluation study

Report on the final user evaluation study. This deliverable is produced by Task T1.3.

An overview of resources in the making

Resulting from T92

Overview of datasets for the SLs of Europe

This deliverable results from T61 It provides an overview of available datasets for European SLs also providing size and access information The report also gives an overview of grammatical patterns as occurring in the European SLs where available providing statistical measures on these from the available datasets

Report on performance metrics and user study preparations

Report on the definition of objective and subjective performance metrics for single components and the overall EASIER system also including recruitment strategies This deliverable is produced by Task T12

Definition of minimal contents of data set for participation

Resulting from T91

A tool allowing post-editing of text

This piece of software will include the necessary interface for human post-editors to review and correct NMT text output. This deliverable is produced by T5.1.

Extended interlingual index for the project's core SLs and languages covered in WP9

This step extends the results from D6.3.2 as far as possible to further languages as documented in WP9 (D9.4-6). It is the final activity in T6.2.

Translation models, final version

This deliverable consists of a sign-to-spoken/spoken-to-sign translation model that accepts as input/output five languages and a spoken-to-spoken translation model that accepts as input/output six languages (see Table 4). It is produced by Tasks 4.2 and 4.3.

Tools for harmonizing broadcast text data

This collection of open-source tools converts from the formats identified in D6.5 to a common format related to D6.2. Its delivery concludes T6.3.

Initial interlingual index for DGS and BSL

The dataset relates lexical data available in the DGS and BSL datasets identified in D6.1 by means of a concept-based interlingual index. It is part of the activities in T6.2.

A stand-alone editor for discourse representation diagrams

This piece of software will offer a graphical interface allowing to edit discourse representations through the graphical representation defined in T5.3.1. This deliverable is produced by Tasks 5.3.1 and 5.3.2.

Text analysis modules

This is a piece of software developed as a basis for rule-based postprocessing of the NMT output when going from spoken language to SL. This deliverable is produced by Task 4.4.

Quality estimation models for German and DGS

These are two pieces of software to predict the quality of automatic translations from German into DGS and vice versa to inform the human post-editors. This deliverable is produced by Task 5.4.

Tools for harmonizing available annotations to a common format

This deliverable within T6.2 follows D6.2 and provides open-source software solution for converters of (linguistic) annotation into a common interchange format.

Final integration allowing post-editing with discourse representation diagrams

This piece of software will integrate the graphical editor of D5.3 and pass on the generated output to the Paula animation system. This deliverable is produced by Tasks 5.3.3 and 5.3.4.

Interlingual index for the project's core SLs

This dataset extends D6.3.1 to the project’s core SLs, by extracting or constructing semi-automatically from the datasets identified in D6.1. It is part of the T6.2 results.

Establishment of the infrastructure to automatically analyse other datasets

Setup of a hardware- and software infrastructure to process data on European SLs outside EASIER’s core, as the initial step in T9.4.

A tool allowing post-editing of SL with human SL input

This extension to the post-editing tool of D5.1 will integrate the necessary functionality for human post-editors to review and correct SL animations using a webcam. This deliverable is produced by Task 5.2.

Data Management Plan V1

This document to be updated at the end of each reporting period D113 D114 will identify the best practices and specific standards for the generated data assess their suitability for sharingreuse in accordance with official EC guidelines describe the plan for the IPR management 1st version and its implementation 2nd and final version This deliverable is produced by Task 113 and gets input from Task 112

Data Management Plan V3

The final update of the document will identify the best practices and specific standards for the generated data, assess their suitability for sharing/reuse in accordance with official EC guidelines, describe the plan for the IPR management and its implementation (final version). This deliverable is produced by Task 11.3 and gets input from Task 11.2.

Data Management Plan V2

The second update of the document will identify the best practices and specific standards for the generated data assess their suitability for sharingreuse in accordance with official EC guidelines describe the plan for the IPR management and its implementation 2nd version This deliverable is produced by Task 113 and gets input from Task 112

Workshop and training for experts in under-resourced languages Session 2

Is offline teaching based on D9.3 as part of the T9.3 activities in order to extend the scope of EASIER to more European SLs.

Workshop and training for experts in under-resourced languages and neologisms Session 1

Is offline teaching based on D9.3 as part of the T9.3 activities in order to extend the scope of EASIER to more European SLs.

Publicaciones

Hierarchical I3D for Sign Spotting

Autores: Wong, Ryan; Camgöz, Necati Cihan; Bowden, Richard
Publicado en: Computer Vision – ECCV 2022 Workshops, 2022
Editor: Lecture Notes in Computer Science
DOI: 10.48550/arxiv.2210.00951

Changing the Representation: Examining Language Representation for Neural Sign Language Production

Autores: Harry Walsh, Ben Saunders, Richard Bowden
Publicado en: Proceedings of the LREC 2022 7th Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual & the Textual Challenges and Perspectives (SLTAT 7), 2022, Página(s) 117-124, ISBN 979-10-95546-82-5
Editor: European Language Resources Association (ELRA)

Representation and Synthesis of Geometric Relocations

Autores: Michael Filhol and John C. McDonald
Publicado en: Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, 2022, Página(s) 53-58, ISBN 979-10-95546-86-3
Editor: European Language Resources Association (ELRA)

Reducing Gender Bias in NMT with FUDGE

Autores: Lu, Tianshuai; Aepli, Noëmi; Rios, Annette
Publicado en: Proceedings of the First Workshop on Gender-Inclusive Translation Technologies, 2023, Página(s) 61--69
Editor: European Association for Machine Translation
DOI: 10.5167/uzh-234532

A Comparative Cross Language View On Acted Databases Portraying Basic Emotions Utilising Machine Learning

Autores: Felix Burkhardt, Anabell Hacker, Uwe Reichel, Hagen Wierstorf, Florian Eyben and Björn Schuller
Publicado en: Proceedings of the 13th Language Resources and Evaluation Conference, 2022, Página(s) 1917–1924
Editor: European Language Resources Association (ELRA)

Introducing Sign Languages to a Multilingual Wordnet: Bootstrapping Corpora and Lexical Resources of Greek Sign Language and German Sign Language

Autores: Sam Bigeard, Marc Schulder, Maria Kopf, Thomas Hanke, Kiki Vasilaki, Anna Vacalopoulou, Theodoros Goulas, Athanasia-Lida Dimou, Stavroula-Evita Fotinea, Eleni Efthimiou
Publicado en: Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, 2022, Página(s) 9-15, ISBN 979-10-95546-86-3
Editor: European Language Resources Association (ELRA)

The Sign Language Dataset Compendium: Creating an Overview of Digital Linguistic Resources

Autores: Maria Kopf, Marc Schulder, Thomas Hanke
Publicado en: Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, 2022, Página(s) 102-109, ISBN 979-10-95546-86-3
Editor: European Language Resources Association (ELRA)

AfriSign: Machine Translation for African Sign Languages

Autores: Gueuwou Shester, Takyi Kate, Müller Mathias, Nyarko Marco Stanley, Adade,Richard, Gyening Rose-Mary Owusuaa Mensah
Publicado en: 4th Workshop on African Natural Language Processing, 2023
Editor: Machine Translation for African Sign Languages
DOI: 10.5167/uzh-237746

Facilitating the Spread of New Sign Language Technologies across Europe

Autores: Morgan, Hope E., Crasborn, Onno, Kopf, Maria, Schulder, Marc, Hanke, Thomas
Publicado en: Proceedings of the LREC2022 10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, 2022, Página(s) 144-172, ISBN 979-10-95546-86-3
Editor: European Language Resources Association (ELRA)

Anonysign: Novel Human Appearance Synthesis for Sign Language Video Anonymisation

Autores: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
Publicado en: 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021
Editor: IEEE
DOI: 10.1109/fg52635.2021.9666984

Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition

Autores: P. Antoniadis, P. P. Filntisis and P. Maragos
Publicado en: 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021, Página(s) 1-8, ISBN 978-1-6654-3177-4
Editor: IEEE
DOI: 10.1109/fg52635.2021.9667014

Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting

Autores: Jiang, Zifan; Moryossef, Amit; Müller, Mathias; Ebling, Sarah
Publicado en: Findings of the Association for Computational Linguistics: EACL 2023, 2023, Página(s) 1706--1724
Editor: Association for Computational Linguistics
DOI: 10.18653/v1/2023.findings-eacl.127

First WMT Shared Task on Sign Language Translation (WMT-SLT22)

Autores: Müller, Mathias; Ebling, Sarah; Avramidis, Eleftherios; Battisti, Alessia; Berger, Michèle; Bowden, Richard; Braffort, Annelies; Camgöz, Necati Cihan; Espana-Bonet, Cristina; Grundkiewicz, Roman; Jiang, Zifan; Koller, Oscar; Moryossef, Amit; Perrollaz, Regula; Reinhard, Sabine; Rios, Annette; Shterionov, Dimitar; Sidler-Miserez, Sandra; Tissi, Katja; Van Landuyt, Davy
Publicado en: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023, Página(s) 507--508
Editor: European Association for Machine Translation

Multimodal Recognition of Valence, Arousal and Dominance via Late-Fusion of Text, Audio and Facial Expressions

Autores: Annette Rios, Uwe Reichel, Chirag Bhuvaneshwara, Panagiotis Filntisis, Petros Maragos, Felix Burkhardt, Florian Eyben, Björn Schuller, Fabrizio Nunnari, Sarah Ebling
Publicado en: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning., 2023, ISBN 978-2-87587-088-9
Editor: i6doc.com
DOI: 10.14428/esann/2023.es2023-128

Easier Notation – a Proposal for a Gloss-Based Scripting Language for Sign Language Generation Based on Lexical Data

Autores: Hanke, Thomas; König, Lutz; Konrad, Reiner; Kopf, Maria; Schulder, Marc; Wolfe, Rosalee
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops: Sign Language Translation and Avatar Technology, 2023
Editor: IEEE
DOI: 10.1109/icasspw59220.2023.10192997

Organizing a Bilingual Lexicographic Database with the Use of WordNet

Autores: A. Vacalopoulou, E. Efthimiou, S.-E. Fotinea, T. Goulas, A.-L. Dimou and K. Vasilaki
Publicado en: Proceedings of XX EURALEX International Congress: Dictionaries and Society, 2022
Editor: tbd

Improving 3D Pose Estimation For Sign Language

Autores: Ivashechkin, Maksym; Mendez, Oscar; Bowden, Richard
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Editor: IEEE
DOI: 10.1109/icasspw59220.2023.10193629

Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production

Autores: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
Publicado en: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, Página(s) 5141-5151
Editor: Computer Vision Foundation

Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wild

Autores: I. Pikoulis, P. P. Filntisis and P. Maragos
Publicado en: 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021, Página(s) 01-08, ISBN 978-1-6654-3177-4
Editor: IEEE
DOI: 10.1109/fg52635.2021.9666957

Signing Avatar Performance Evaluation within the EASIER Project

Autores: A. Dimou, V. Papavassiliou, J. McDonald, T. Goulas, K. Vasilaki, A. Vacalopoulou, S-E Fontinea, E. Efthimiou, R. Wolfe
Publicado en: Seventh International Workshop on Sign Language Translation and Avatar Technology (SLTAT), 2022
Editor: ELRA

Language Accessibility for the Deaf and Hard-of-Hearing

Autores: M. Debevc, R. Wolfe, S. Ebling
Publicado en: Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE), 2022
Editor: tbd

Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Production

Autores: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
Publicado en: Proceedings of the 7th Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual & the Textual Challenges and Perspectives (SLTAT 7), 2022, Página(s) 95-102, ISBN 979-10-95546-82-5
Editor: European Language Resources Association (ELRA)

A software toolkit for pre-processing sign language video streams

Autores: F. Nunnari
Publicado en: Seventh International Workshop on Sign Language Translation and Avatar Technology (SLTAT), 2022
Editor: ELRA

Fine-tuning of convolutional neural networks for the recognition of facial expressions in sign language video samples

Autores: N. Deshpande, F. Nunnari & E. Avramidis
Publicado en: Seventh International Workshop on Sign Language Translation and Avatar Technology (SLTAT), 2022
Editor: ELRA

Findings of the First WMT Shared Task on Sign Language Translation (WMT-SLT22)

Autores: Müller, Mathias; Ebling, Sarah; Avramidis, Eleftherios; Battisti, Alessia; Berger, Michèle; Bowden, Richard; Braffort, Annelies; Cihan Camgöz, Necati; España-Bonet, Cristina; Grundkiewicz, Roman; Jiang, Zifan; Koller, Oscar; Moryossef, Amit; Perrollaz, Regula; Reinhard, Sabine; Rios, Annette; Shterionov, Dimitar; Sidler-Miserez, Sandra; Tissi, Katja; Van Landuyt, Davy
Publicado en: Seventh Conference on Machine Translation (WMT), 2023, Página(s) 744–-772
Editor: Association for Computational Linguistics
DOI: 10.5167/uzh-227008

SyntAct: A Synthesized Database of Basic Emotions

Autores: Felix Burkhardt, Florian Eyben and Björn W. Schuller
Publicado en: Proceedings of the 1st Workshop on Dataset Creation for Lower-Resourced Languages (DCLRL) @LREC2022, 2022, Página(s) 1-9
Editor: European Language Resources Association (ELRA)

Gloss Alignment using Word Embeddings

Autores: Walsh, Harry; Sincan, Ozge Mercanoglu; Saunders, Ben; Bowden, Richard
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Editor: IEEE
DOI: 10.1109/icasspw59220.2023.10193013

Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives

Autores: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
Publicado en: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, Página(s) 1919-1929, ISBN 978-1-6654-2812-5
Editor: IEEE
DOI: 10.1109/iccv48922.2021.00193

Representation Matters: The Case for Diversifying Sign Language Avatars

Autores: M. Kopf, R. Omardeen and D. Van Landuyt
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023, ISBN 979-8-3503-0261-5
Editor: IEEE
DOI: 10.25592/uhhfdm.13758

Supporting Mouthing in Sign Languages: New Innovations and a Proposal for Future Corpus Building

Autores: R. Wolfe, J. McDonald, R. Johnson, B. Stuff, S. Klinghoffer, A. Bonzani, A. Alexander, N. Barnekow
Publicado en: Seventh International Workshop on Sign Language Translation and Avatar Technology (SLTAT), 2022
Editor: ELRA

The Sign Language Interchange Format: Harmonising Sign Language Datasets For Computational Processing

Autores: Schulder, Marc; Bigeard, Sam; Hanke, Thomas; Kopf, Maria
Publicado en: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops: Sign Language Translation and Avatar Technology, 2023
Editor: IEEE
DOI: 10.1109/icasspw59220.2023.10193022

Nkululeko: A Tool For Rapid Speaker Characteristics Detection

Autores: Felix Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben and Björn Schuller
Publicado en: Proceedings of the 13th Language Resources and Evaluation Conference, 2022, Página(s) 1925‑1932
Editor: European Language Resources Association (ELRA)

Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering

Autores: G. Rochette, C. Russell and R. Bowden
Publicado en: 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021, Página(s) 1-8, ISBN 978-1-6654-3177-4
Editor: IEEE
DOI: 10.1109/fg52635.2021.9667033

An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation

Autores: Moryossef, Amit; Müller, Mathias; Göhring, Anne; Jiang, Zifan; Goldberg, Yoav; Ebling, Sarah
Publicado en: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023, Página(s) 22-33
Editor: European Association for Machine Translation
DOI: 10.48550/arxiv.2305.17714

Considerations for meaningful sign language machine translation based on glosses

Autores: Müller, Mathias; Jiang, Zifan; Moryossef, Amit; Rios, Annette; Ebling, Sarah
Publicado en: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2023, Página(s) 682--693
Editor: Association for Computational Linguistics
DOI: 10.18653/v1/2023.acl-short.60

SLTUNET: A Simple Unified Model for Sign Language Translation

Autores: Biao Zhang, Mathias Müller, Rico Sennrich
Publicado en: 2023
Editor: The Eleventh International Conference on Learning Representations
DOI: 10.5167/uzh-234791

The Myth of Signing Avatars

Autores: John C. McDonald, Rosalee Wolfe, Eleni Efthimiou, Evita Fontinea, Frankie Picron, Davy Van Landuyt, Tina Sioen, Annelies Braffort, Michael Filhol, Sarah Ebling, Thomas Hanke, and Verena Krausneker
Publicado en: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021, Página(s) 33-42
Editor: Association for Machine Translation in the Americas

What about synthetic signing? A methodology for signer involvement in the development of avatar technology with generative capacity

Autores: Dimou A-L, Papavassiliou V, Goulas T, Vasilaki K, Vacalopoulou A, Fotinea S-E and Efthimiou E
Publicado en: Frontiers in communication, Edición 2297900X, 2022, ISSN 2297-900X
Editor: Frontiers in communication
DOI: 10.3389/fcomm.2022.798644

Sign Language Avatars: A Question of Representation

Autores: Wolfe, Rosalee, John C. McDonald, Thomas Hanke, Sarah Ebling, Davy Van Landuyt, Frankie Picron, Verena Krausneker, Eleni Efthimiou, Evita Fotinea, and Annelies Braffort
Publicado en: Special Edición Frontiers in Machine Translation, Edición 13, 2022, Página(s) 206, ISSN 2078-2489
Editor: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13040206

Speech-based Age and Gender Prediction with Transformers

Autores: Burkhardt, Felix; Wagner, Johannes; Wierstorf, Hagen; Eyben, Florian; Schuller, Björn
Publicado en: Speech Communication - 15th ITG Conference, 2023, ISSN 0340-4161
Editor: V D E Verlag GmbH
DOI: 10.48550/arxiv.2306.16962

Best practices for sign language technology research

Autores: Neil Fox, Bencie Woll, Kearsy Cormier
Publicado en: Universal Access in the Information Society, 2023, ISSN 1615-5289
Editor: Springer Verlag
DOI: 10.1007/s10209-023-01039-1

Buscando datos de OpenAIRE...

Resultado final

Publicaciones

Compartir esta página

Descargar