Voice driven interaction in XR spaces

Project Information

VOXReality

Grant agreement ID: 101070521

DOI

10.3030/101070521

Project closed

EC signature date 11 June 2022

Start date 1 October 2022

End date 31 December 2025

Funded under

• Digital, Industry and Space

Total cost

€ 4 786 875,00

EU contribution

€ 4 786 875,00

Coordinated by

MAGGIOLI SPA
Italy

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Documents, reports (6)

Dissemination and communication plan

This report will outline communication and dissemination channels to be used and identify relevant events and publications.

Development infrastructure and integration guidelines

Analysis of the integration and validation requirements

Model deployment analysis V1

This deliverable will analyse the progress and results on VOXReality models deployment, training and XR applications.

Definition and Analysis of VOXReality Use Cases V1

Detailed analysis of use cases, technical and user requirements for each case of the project.

Advanced AI multi-modal for XR analysis V1

This deliverable will analyse the progress and results on VOXReality Audio processing, VOXReality Multilingual translation, Vision and language based XR models and Context aware generative dialogue system.

Open call documents collection V1

This deliverable will contain all the needed information and necessary documents for the applicants to submit applications to the Open Call.

Publications

Peer reviewed articles (2)

Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues

Author(s): Wentao Deng, Jiahuan Pei, Zhaochun Ren, Zhumin Chen, Pengjie Ren
Published in: Transactions of the Association for Computational Linguistics, Issue 11, 2023, ISSN 2307-387X
Publisher: MIT Press
DOI: 10.1162/TACL_A_00599

SADAMB: Advancing Spatially-Aware Vision-Language Modeling Through Datasets, Metrics, and Benchmarks

Author(s): Giorgos Papadopoulos, Petros Drakoulis, Athanasios Ntovas, Alexandros Doumanoglou, Dimitris Zarpalas
Published in: Computers, Issue 14, 2025, ISSN 2073-431X
Publisher: MDPI AG
DOI: 10.3390/COMPUTERS14100413

Conference proceedings (11)

VOXReality: Immersive XR experiences

Author(s): Maniatis, A., Bourou, S., Anastasakis, Z., Psychogios, K.
Published in: IHIET-AI 2023, Issue 70, 2023
Publisher: AHFE (2023) International Conference. AHFE Open Access

Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models

Author(s): Paweł Mąka, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Published in: Proceedings of the 31st International Conference on Computational Linguistics, 2025
Publisher: Association for Computational Linguistics

Enhancing the Audience Experience for VR and AR Theatre with AI-generated Subtitles

Author(s): Irene Viola, Moonisa Ahsan, Olga Chatzifoti, Atanas Yonkov, Eleni Oikonomou, Ioannis Radin, Paweł Mąka, Abderrahmane Issam, Pablo Cesar
Published in: Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology, 2025
Publisher: ACM
DOI: 10.1145/3756884.3766057

You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models

Author(s): Pawel Maka, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Published in: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025
Publisher: Association for Computational Linguistics
DOI: 10.18653/V1/2025.EMNLP-MAIN.1394

Sequence Shortening for Context-Aware Machine Translation

Author(s): Mąka, P., Semerci, Y. C., Scholtes, J., & Spanakis, G.
Published in: Findings of the Association for Computational Linguistics: EACL 2024, 2024
Publisher: Association for Computational Linguistics
DOI: 10.48550/ARXIV.2402.01416

User centric Requirements for Enhancing XR Use Cases with Machine Learning Capabilities

Author(s): Sueyoon Lee, Moonisa Ahsan, Irene Viola, Pablo Cesar
Published in: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2025
Publisher: IEEE
DOI: 10.1109/VRW66409.2025.00152

VRD: A multi-lingual translation and Ai-Assisted Navigation experience for VR Conference Application

Author(s): Moonisa Ahsan, Irene Viola, Manuel Toledo, Dimitris Kontopoulos, Pablo Cesar
Published in: Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology, 2025
Publisher: ACM
DOI: 10.1145/3756884.3768418

Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality

Author(s): Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar
Published in: 62nd Annual Meeting of the Association for Computational Linguistics, 2024
Publisher: ACL 2024
DOI: 10.48550/ARXIV.2405.13034

Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters

Author(s): Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Published in: Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), 2024
Publisher: Association for Computational Linguistics
DOI: 10.18653/V1/2024.IWSLT-1.36

DTW-Align: Bridging the Modality Gap in End-to-End Speech Translation with Dynamic Time Warping Alignment

Author(s): Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Published in: Proceedings of the Tenth Conference on Machine Translation, 2025
Publisher: Association for Computational Linguistics
DOI: 10.18653/V1/2025.WMT-1.11

A Representation Level Analysis of NMT Model Robustness to Grammatical Errors

Author(s): Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
Published in: Findings of the Association for Computational Linguistics: ACL 2025, 2025
Publisher: Association for Computational Linguistics
DOI: 10.18653/V1/2025.FINDINGS-ACL.451

Book chapters (1)

Using Augmented Reality and Machine Learning for Captioning in Theatrical Experiences

Author(s): Olga Chatzifoti, Nick Achilleopoulos, Spiros Borotis, Moonisa Ahsan, and Eleni Oikonomou
Published in: EuroXR 2024: Proceedings of the 21st EuroXR International Conference, Issue 322, 2024, ISSN 2242-1211
Publisher: VTT Technical Research Centre of Finland
DOI: 10.32040/2242-122X.2024.T432