aRTIFICIAL iNTELLIGENCE for the Deaf

Informazioni relative al progetto

aiD

ID dell’accordo di sovvenzione: 872139

DOI

10.3030/872139

Progetto chiuso

Data della firma CE 20 Novembre 2019

Data di avvio 1 Dicembre 2019

Data di completamento 30 Novembre 2023

Finanziato da

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Costo totale

€ 1 587 000,00

Contributo UE

€ 1 587 000,00

Coordinato da

TECHNOLOGIKO PANEPISTIMIO KYPROU
Cyprus

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Risultati finali

Documents, reports (3)

Compression algorithms

Network compression leveraging i the Bayesian inference arguments that underly the envisaged machine learning models of WP4 and WP5 ii network distillation approaches

Generative system translating language to SL gesture trajectories

The algorithms developed for translating language (speech/text) to SL trajectories, trained using the dataset of D5.1, and embedded into an AR environment.

System translating SL footage to language

The Deep Learning models aiD will devise to address the problem of generating text transcription and synthetic speech pertaining to SL video footage trained using the dataset of D41

Data sets, microdata, etc (2)

Curated dataset for learning to generate avatars depicting SL

We will process the dataset obtained in deliverable D4.1 using tracking algorithms to obtain pairs of text sequences and corresponding hand, arm, lip and upper body motion trajectories. This is needed for training deep learning models capable of generating motion trajectories of moving avatars that can function as virtual interpreters of language to SL (D5.2).

Curated dataset for learning to translate SL footage

This is the processed dataset that will be used for the development/training of machine learning models translating SL footage. This dataset will be obtained by processing existing raw video footage of SL, that is publicly available and will be provided by ERT and PTV. Fresh raw footage may also be generated by staff members of HUP (professional SL translators), if needed.

Demonstrators, pilots, prototypes (2)

Realization of pilot software

A software implementation of the three envisaged demos/pilots.

Pilot Deployment and Evaluation

"This deliverable deals with the deployment in a real-world setting of the three pilots, and their evaluation by (anonymous) volunteer users (deaf individuals) or professional SL interpreters.Specifically, regarding the AR service: HFD and EUD will enrol volunteer SL users who will act as the evaluators of our demonstrator; HFD and EUD will achieve this by reaching out to their members. The volunteers will spend some time using the developed AR news service, and will provide us feedback on both the quality of the AR-based SL footage (accuracy, consistency) and the real-time behaviour of the solution (e.g., computational lags that hinder real-time performance, computational requirements, etc.). To this end, HDR and EUD will develop appropriate questionnaires. The enrolled volunteers will perform evaluation using portable devices provided by aiD, which will have aiD software installed and running. Regarding the automated Relay Service prototype: HFD will enrol volunteer SL users who will act as the evaluators of our demonstrator; HFD will achieve this by reaching out to its members. The volunteers will be asked to use the Relay Service in specific mock-up scenarios, designed by ANT and implemented in the premises of ANT. Then, the volunteers will provide us feedback on the quality and timeliness of the service: that is, accuracy and consistency in interpreting what they ""say"" in SL to the hearing operators, as well as time needed for the system to (correctly) perform the interpretation task. The evaluation will be performed using appropriate questionnaires developed by ANT.Regarding the Interactive Digital Tutor prototype:HFD will take the lead in developing some teaching materials for first-grade deaf children, using their members of staff who are special education teachers. These are some simple reading material for first-graders and the corresponding foundational SL primitives needed for their translation. Subsequently, HDR will use the outcomes of WP4-WP6 so as to develop an automated interactive tutoring system for this material. The main principle behind the envisaged pilot is that the user will be able to ask the system repeat a word, a phrase, or a larger excerpt they want to see again translated in SL gestures. In addition, the user will be able to perform some SL gestures (similar to what they have learned through the system) and see how these are interpreted in SL. Evaluation will be performed by having HFD and EUD members of staff provide us feedback on the quality of the teaching system. We focus on both the capacity of its interactive functionally to improve the learning outcomes for deaf first-graders, as well as its translation accuracy and computational speed/responsiveness."

Websites, patent fillings, videos etc. (1)

Website and social media presence

A public website and accounts in social media Publicity material including videos describing the project rationale and ambition

Pubblicazioni

Conference proceedings (9)

Dialog speech sentiment classification for imbalanced datasets

Autori: Sergis Nicolaou, Lambros Mavrides, Georgina Tryfou, Kyriakos Tolias, Konstantinos Panousis, Sotirios Chatzis, Sergios Theodoridis
Pubblicato in: Proceedings of SPECOM 2021, 2021
Editore: Springer Nature

Variational Conditional Dependence Hidden Markov Models for Skeleton-Based Action Recognition

Autori: Panousis, Konstantinos Panagiotis; Chatzis, Sotirios; Theodoridis, Sergios
Pubblicato in: Proceedings of ISVC 2021, 2021
Editore: Springer

Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness

Autori: Konstantinos P. Panousis, Sotirios Chatzis, and Sergios Theodoridis
Pubblicato in: NeurIPS Workshops 2022, 2021
Editore: NeurIPS
DOI: 10.5281/zenodo.6000329

A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset

Autori: Andreas Voskou, Konstantinos P. Panousis, Harris Partaourides, Kyriakos Tolias, Sotirios Chatzis
Pubblicato in: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. p. 1966-1975, 2023
Editore: ICCVW2023 - ACVR
DOI: 10.48550/arxiv.2310.04753

Local Competition and Stochasticity for Adversarial Robustness in Deep Learning

Autori: Konstantinos Panousis, Sotirios Chatzis, Antonios Alexos, Sergios Theodoridis
Pubblicato in: Proceedings of AISTATS 2021, 2021
Editore: PMLR
DOI: 10.5281/zenodo.5498188

Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representations

Autori: Konstantinos P. Panousis, Anastasios Antoniadis, Sotirios Chatzis
Pubblicato in: Proceedings of AAAI 2022, 2022
Editore: AAAI
DOI: 10.5281/zenodo.6000363

Making Vision Networks Interpretable viaCompetition and Dissection

Autori: Konstantinos Panousis, Sotirios Chatzis
Pubblicato in: Proceedings of NeurIPS 2023, 2023
Editore: Konstantinos Panousis
DOI: 10.48550/arxiv.2310.04929

Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translation

Autori: Voskou, Andreas; Panousis, Konstantinos P.; Kosmopoulos, Dimitrios; Metaxas, Dimitris N.; Chatzis, Sotirios
Pubblicato in: Proc. ICCV 2021, 2021
Editore: ICCV
DOI: 10.5281/zenodo.5498338

Stochastic Deep Networks with Linear Competing Units for Model-Agnostic Meta-Learning

Autori: Konstantinos Kalais, Sotirios Chatzis
Pubblicato in: Proc. ICML 2022, 2022
Editore: ICML
DOI: 10.5281/zenodo.6580332