aRTIFICIAL iNTELLIGENCE for the Deaf

Projektinformationen

aiD

ID Finanzhilfevereinbarung: 872139

DOI

10.3030/872139

Projekt abgeschlossen

EK-Unterschriftsdatum 20 November 2019

Startdatum 1 Dezember 2019

Enddatum 30 November 2023

Finanziert unter

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Gesamtkosten

€ 1 587 000,00

EU-Beitrag

€ 1 587 000,00

Koordiniert durch

TECHNOLOGIKO PANEPISTIMIO KYPROU
Cyprus

CORDIS bietet Links zu öffentlichen Ergebnissen und Veröffentlichungen von HORIZONT-Projekten.

Links zu Ergebnissen und Veröffentlichungen von RP7-Projekten sowie Links zu einigen Typen spezifischer Ergebnisse wie Datensätzen und Software werden dynamisch von OpenAIRE abgerufen.

Leistungen

Documents, reports (3)

Compression algorithms

Network compression leveraging i the Bayesian inference arguments that underly the envisaged machine learning models of WP4 and WP5 ii network distillation approaches

Generative system translating language to SL gesture trajectories

The algorithms developed for translating language (speech/text) to SL trajectories, trained using the dataset of D5.1, and embedded into an AR environment.

System translating SL footage to language

The Deep Learning models aiD will devise to address the problem of generating text transcription and synthetic speech pertaining to SL video footage trained using the dataset of D41

Data sets, microdata, etc (2)

Curated dataset for learning to generate avatars depicting SL

We will process the dataset obtained in deliverable D4.1 using tracking algorithms to obtain pairs of text sequences and corresponding hand, arm, lip and upper body motion trajectories. This is needed for training deep learning models capable of generating motion trajectories of moving avatars that can function as virtual interpreters of language to SL (D5.2).

Curated dataset for learning to translate SL footage

This is the processed dataset that will be used for the development/training of machine learning models translating SL footage. This dataset will be obtained by processing existing raw video footage of SL, that is publicly available and will be provided by ERT and PTV. Fresh raw footage may also be generated by staff members of HUP (professional SL translators), if needed.

Demonstrators, pilots, prototypes (2)

Realization of pilot software

A software implementation of the three envisaged demos/pilots.

Pilot Deployment and Evaluation

"This deliverable deals with the deployment in a real-world setting of the three pilots, and their evaluation by (anonymous) volunteer users (deaf individuals) or professional SL interpreters.Specifically, regarding the AR service: HFD and EUD will enrol volunteer SL users who will act as the evaluators of our demonstrator; HFD and EUD will achieve this by reaching out to their members. The volunteers will spend some time using the developed AR news service, and will provide us feedback on both the quality of the AR-based SL footage (accuracy, consistency) and the real-time behaviour of the solution (e.g., computational lags that hinder real-time performance, computational requirements, etc.). To this end, HDR and EUD will develop appropriate questionnaires. The enrolled volunteers will perform evaluation using portable devices provided by aiD, which will have aiD software installed and running. Regarding the automated Relay Service prototype: HFD will enrol volunteer SL users who will act as the evaluators of our demonstrator; HFD will achieve this by reaching out to its members. The volunteers will be asked to use the Relay Service in specific mock-up scenarios, designed by ANT and implemented in the premises of ANT. Then, the volunteers will provide us feedback on the quality and timeliness of the service: that is, accuracy and consistency in interpreting what they ""say"" in SL to the hearing operators, as well as time needed for the system to (correctly) perform the interpretation task. The evaluation will be performed using appropriate questionnaires developed by ANT.Regarding the Interactive Digital Tutor prototype:HFD will take the lead in developing some teaching materials for first-grade deaf children, using their members of staff who are special education teachers. These are some simple reading material for first-graders and the corresponding foundational SL primitives needed for their translation. Subsequently, HDR will use the outcomes of WP4-WP6 so as to develop an automated interactive tutoring system for this material. The main principle behind the envisaged pilot is that the user will be able to ask the system repeat a word, a phrase, or a larger excerpt they want to see again translated in SL gestures. In addition, the user will be able to perform some SL gestures (similar to what they have learned through the system) and see how these are interpreted in SL. Evaluation will be performed by having HFD and EUD members of staff provide us feedback on the quality of the teaching system. We focus on both the capacity of its interactive functionally to improve the learning outcomes for deaf first-graders, as well as its translation accuracy and computational speed/responsiveness."

Websites, patent fillings, videos etc. (1)

Website and social media presence

A public website and accounts in social media Publicity material including videos describing the project rationale and ambition

Veröffentlichungen

Conference proceedings (9)

Dialog speech sentiment classification for imbalanced datasets

Autoren: Sergis Nicolaou, Lambros Mavrides, Georgina Tryfou, Kyriakos Tolias, Konstantinos Panousis, Sotirios Chatzis, Sergios Theodoridis
Veröffentlicht in: Proceedings of SPECOM 2021, 2021
Herausgeber: Springer Nature

Variational Conditional Dependence Hidden Markov Models for Skeleton-Based Action Recognition

Autoren: Panousis, Konstantinos Panagiotis; Chatzis, Sotirios; Theodoridis, Sergios
Veröffentlicht in: Proceedings of ISVC 2021, 2021
Herausgeber: Springer

Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness

Autoren: Konstantinos P. Panousis, Sotirios Chatzis, and Sergios Theodoridis
Veröffentlicht in: NeurIPS Workshops 2022, 2021
Herausgeber: NeurIPS
DOI: 10.5281/zenodo.6000329

A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset

Autoren: Andreas Voskou, Konstantinos P. Panousis, Harris Partaourides, Kyriakos Tolias, Sotirios Chatzis
Veröffentlicht in: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. p. 1966-1975, 2023
Herausgeber: ICCVW2023 - ACVR
DOI: 10.48550/arxiv.2310.04753

Local Competition and Stochasticity for Adversarial Robustness in Deep Learning

Autoren: Konstantinos Panousis, Sotirios Chatzis, Antonios Alexos, Sergios Theodoridis
Veröffentlicht in: Proceedings of AISTATS 2021, 2021
Herausgeber: PMLR
DOI: 10.5281/zenodo.5498188

Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representations

Autoren: Konstantinos P. Panousis, Anastasios Antoniadis, Sotirios Chatzis
Veröffentlicht in: Proceedings of AAAI 2022, 2022
Herausgeber: AAAI
DOI: 10.5281/zenodo.6000363

Making Vision Networks Interpretable viaCompetition and Dissection

Autoren: Konstantinos Panousis, Sotirios Chatzis
Veröffentlicht in: Proceedings of NeurIPS 2023, 2023
Herausgeber: Konstantinos Panousis
DOI: 10.48550/arxiv.2310.04929

Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translation

Autoren: Voskou, Andreas; Panousis, Konstantinos P.; Kosmopoulos, Dimitrios; Metaxas, Dimitris N.; Chatzis, Sotirios
Veröffentlicht in: Proc. ICCV 2021, 2021
Herausgeber: ICCV
DOI: 10.5281/zenodo.5498338

Stochastic Deep Networks with Linear Competing Units for Model-Agnostic Meta-Learning

Autoren: Konstantinos Kalais, Sotirios Chatzis
Veröffentlicht in: Proc. ICML 2022, 2022
Herausgeber: ICML
DOI: 10.5281/zenodo.6580332