Socially-acceptable Extended Reality Models and Systems

Informacje na temat projektu

SERMAS

Identyfikator umowy o grant: 101070351

DOI

10.3030/101070351

Projekt został zamknięty

Data podpisania przez KE 20 Czerwca 2022

Data rozpoczęcia 1 Października 2022

Data zakończenia 31 Października 2025

Finansowanie w ramach

• Digital, Industry and Space

Koszt całkowity

€ 4 444 228,75

Wkład UE

€ 4 444 228,75

Koordynowany przez

UNIVERSITA DEGLI STUDI DI MODENA E REGGIO EMILIA
Italy

CORDIS oferuje możliwość skorzystania z odnośników do publicznie dostępnych publikacji i rezultatów projektów realizowanych w ramach programów ramowych HORYZONT.

Odnośniki do rezultatów i publikacji związanych z poszczególnymi projektami 7PR, a także odnośniki do niektórych konkretnych kategorii wyników, takich jak zbiory danych i oprogramowanie, są dynamicznie pobierane z systemu OpenAIRE .

Rezultaty

Documents, reports (18)

Third-party projects results

Third-party projects results (report on third-party projects activities, achieved results and the impacts (within and beyond SERMAS))

Tutorials and Documentation for the SERMAS toolkit
Periodic progress report for Year 1

Periodic progress report for Year 1.This deliverable will include a section detailing the ethical and legal framework and current compliance of project activities.

Characterization of human mechanisms in interaction processes
Project presentation

Project presentation (includes logo and website, which will be updated regularly)

Analysis of in-lab validation results
Methodologies and tools for user-adapted interaction
Final release of the completed dialogue system
Specification of nonverbal communication software interface
Validation of dialogue system via automatic and human metrics
Security assessement of the SERMAS XR Agent modular architecture
Final project report

Final project report.This deliverable will include a section detailing the ethical and legal framework and current compliance of project activities.

Results of environment digitalization
Periodic progress report for Year 2

Periodic progress report for Year 2.This deliverable will include a section detailing the ethical and legal framework and current compliance of project activities.

Description of the modular architecture of the SERMAS XR Agent and design of its modules

Description of the modular architecture of the SERMAS XR Agent and design of its modules; first release at M6, to be updated at M12 once the detailed description of case studies is available

The SERMAS Methodology
Proof-of-Concept Reports

Proof-of-Concept Reports (updated at months 24 and 36)

Exploitation: lessons learned and best-practices

Data Management Plan (1)

Data management plan

Data management plan (updated regularly)

Demonstrators, pilots, prototypes (1)

Proof-of-Concept Plans

Proof-of-Concept Plans (updated at each iteration, at months 21 and 32)

Other (5)

Second interim release of the nonverbal communication software module
Dialogue system capable of handling natural language inputs and outputs
Next generation dialogue system that also incoroporates non-verbal information
Final release of the validated nonverbal communication software module

Final release of the validated nonverbal communication software module.The Type is actually R&O.

First interim release of the nonverbal communication software module

Data sets, microdata, etc (1)

Datasets suitable for training, validation, and testing of modules, along with infrastructure to support data collection/processing

Websites, patent fillings, videos etc. (1)

SERMAS website

The SERMAS website will be updated regularly, until the end of the project (M36)

Publikacje

Conference proceedings (29)

Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation

Autorzy: Tianyu Yang, Thy Tran, Iryna Gurevych
Opublikowane w: Findings of the Association for Computational Linguistics: EMNLP 2023, 2023
Wydawca: Association for Computational Linguistics
DOI: 10.18653/V1/2023.FINDINGS-EMNLP.313

The KCL-SAIR team's entry to the GENEA Challenge 2023 Exploring Role-based Gesture Generation in Dyadic Interactions: Listener vs. Speaker

Autorzy: Viktor Schmuck, Nguyen Tan Viet Tuyen, Oya Celiktutan
Opublikowane w: International Cconference on Multimodal Interaction, 2025
Wydawca: ACM
DOI: 10.1145/3610661.3616555

An LLM-based Architecture for Socially Intelligent Robot Navigation based on Social Cues

Autorzy: Andrea Ruo1, Jonathan Cacace2, Mag´ı Dalmau-Moreno2, Lorenzo Sabattini1 and Valeria Villani
Wydawca: IEEE
DOI: 10.5281/ZENODO.15648910

A Mixed Reality System for Interaction with Heterogeneous Robotic Systems

Autorzy: Valeria Villani, Beatrice Capelli, Lorenzo Sabattini
Opublikowane w: 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2024
Wydawca: IEEE
DOI: 10.1109/SMC53992.2023.10394172

User Identification Procedures with Human Mutations: Formal Analysis and Pilot Study

Autorzy: Quamara, Megha; Viganò, Luca
Opublikowane w: 2025 IEEE 22nd International Conference on Software Architecture Companion (ICSA-C), 2025
Wydawca: IEEE
DOI: 10.5281/ZENODO.15168197

Towards More Expressive Human-Robot Interactions: Combining Latent Representations and Diffusion Models for Co-Speech Gesture Generation

Autorzy: Filippo Favali, Viktor Schmuck2, Valeria Villani1, and Oya Celiktutan
Wydawca: Zenodo
DOI: 10.5281/ZENODO.14406640

MASSXR 2025: The 3^rd Workshop on Multi-modal Affective and Social Behavior Analysis and Synthesis in Extended Reality (Affiliated with IEEE VR 2025)

Autorzy: Megha Quamara, Oya Celiktutan, Luca Viganò, Aniket Bera, Pablo Cesar, Funda Durupinar, Aline Normoyle, Chirag Raman, Zerrin Yumak
Opublikowane w: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2025
Wydawca: IEEE
DOI: 10.1109/VRW66409.2025.00066

Exploring the most significant features for EEG ErrP detection through statistical analysis

Autorzy: Alessandra Fava, Valeria Villani, Lorenzo Sabattini
Opublikowane w: 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), 2024
Wydawca: IEEE
DOI: 10.1109/RO-MAN60168.2024.10731260

Follow me: an architecture for user identification and social navigation with a mobile robot

Autorzy: ndrea Ruo, Lorenzo Sabattini, and Valeria Villani
Wydawca: Zenodo
DOI: 10.5281/ZENODO.14246876

Robust CBF-based STL motion planning for socially responsible robot navigation in the presence of measurement noise

Autorzy: Andrea Ruo, Lorenzo Sabattini and Valeria Villani
Wydawca: IEEE
DOI: 10.5281/ZENODO.15648843

Predicting the Intention to Interact with a Service Robot: the Role of Gaze Cues

Autorzy: Simone Arreghini, Gabriele Abbate, Alessandro Giusti, Antonio Paolillo
Opublikowane w: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024
Wydawca: IEEE
DOI: 10.1109/ICRA57147.2024.10610289

User Identification Procedures with Human Mutations: Formal Analysis and Pilot Study

Autorzy: Quamara, Megha,Viganò, Luca
Wydawca: Zenodo
DOI: 10.5281/ZENODO.15168198

A Large Language Model-Based Motion Planning for Human-Robot Interaction: An Experimental Case Study

Autorzy: Andrea Coppari1, Silvia Proia2, Andrea Ruo2, Filippo Favali2, Lorenzo Sabattini2, Cristian Secchi2, Valeria Villani2, Marco Piazzola1, and Luca Capra
Wydawca: Springer
DOI: 10.5281/ZENODO.14259597

CBF-Based Motion Planning for Socially Responsible Robot Navigation Guaranteeing STL Specification^*

Autorzy: Andrea Ruo, Lorenzo Sabattini, Valeria Villani
Opublikowane w: 2024 European Control Conference (ECC), 2024
Wydawca: IEEE
DOI: 10.23919/ECC64448.2024.10591282

Systematic analysis of requirements for socially acceptable service robots

Autorzy: Ruo, Andrea, Arreghini, Simone, Capra, Luca, De Chiara, Rosario, Di Pasquale, Valeria, Giusti, Alessandro, Iani, Cristina, Paolillo, Antonio, Petrak, Dominic, Plaum, Alexander, Quamara, Megha, Sabattini, Lorenzo, Schmuck, Viktor, Servillo, Paolo, Zurolo,
Wydawca: Springer
DOI: 10.5281/ZENODO.14258531

Challenges in Detecting and Analyzing EEG Error-Related Potentials: Lessons from a Case Study in HRI

Autorzy: Alessandra Fava, Adriana Lucchese, Roberto Meattini, Gianluca Palli, Valeria Villani, Lorenzo Sabattini
Opublikowane w: 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), 2024
Wydawca: IEEE
DOI: 10.1109/RO-MAN60168.2024.10731468

Towards a Modular Architecture for eXtended Reality Systems

Autorzy: Quamara, Megha, Capra, Luca, Villani, Valeria, Carlevaro, Cristiano, Celiktutan, Oya, Peretti, Alessandro, Piazzola, Marco, RUO, ANDREA, Sabattini, Lorenzo, Schmuck, Viktor, Viganò, Luca,
Wydawca: Springer
DOI: 10.5281/ZENODO.14276277

The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification

Autorzy: Mohsen Mesgar, Thy Thy Tran, Goran Glavaš, Iryna Gurevych
Opublikowane w: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023
Wydawca: Association for Computational Linguistics
DOI: 10.18653/V1/2023.EACL-MAIN.135

Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues

Autorzy: Dominic Petrak, Thy Thy Tran, Iryna Gurevych
Opublikowane w: Findings of the Association for Computational Linguistics: EMNLP 2024, 2024
Wydawca: Association for Computational Linguistics
DOI: 10.18653/V1/2024.FINDINGS-EMNLP.264

Learning From Free-Text Human Feedback – Collect New Datasets Or Extend Existing Ones?

Autorzy: Dominic Petrak, Nafise Moosavi, Ye Tian, Nikolai Rozanov, Iryna Gurevych
Opublikowane w: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023
Wydawca: Association for Computational Linguistics
DOI: 10.18653/V1/2023.EMNLP-MAIN.1011

Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities

Autorzy: Jiahui Geng, Thy Thy Tran, Preslav Nakov, Iryna Gurevych
Opublikowane w: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025
Wydawca: Association for Computational Linguistics
DOI: 10.18653/V1/2025.ACL-LONG.146

CBF-Based STL Motion Planning for Social Navigation in Crowded Environment

Autorzy: Ruo, Andrea, Sabattini, Lorenzo, Villani, Valeria
Wydawca: Zenodo
DOI: 10.5281/ZENODO.14246898

A Service Robot in the Wild: Analysis of Users Intentions, Robot Behaviors, and Their Impact on the Interaction

Autorzy: Simone Arreghini, Gabriele Abbate, Alessandro Giusti, Antonio Paolillo
Wydawca: IEEE
DOI: 10.48550/ARXIV.2410.03287

AGV Traffic Management in Automated Industrial Plants: An Enhanced Lifelong Multi-Agent Path Finding Approach

Autorzy: Alessandro Bonetti, Silvia Proia, Simone Guidetti, Lorenzo Sabattini
Opublikowane w: 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE), 2024
Wydawca: IEEE
DOI: 10.1109/CASE59546.2024.10711842

Hearing the Robot’s Mind: Sonification for Explicit Feedback in Human-Robot Interaction

Autorzy: Simone Arreghini, Antonio Paolillo, Gabriele Abbate, Alessandro Giusti
Opublikowane w: Springer Proceedings in Advanced Robotics, Human-Friendly Robotics 2024, 2025
Wydawca: Springer
DOI: 10.1007/978-3-031-81688-8_4

A Wearable Stereo Vision-Based Obstacle Detection System for Visually Impaired Individuals

Autorzy: Andrea Ruo, Dario Napolitano, Valeria Villani
Opublikowane w: Communications in Computer and Information Science, Agents and Robots for reliable Engineered Autonomy, 2025
Wydawca: Springer Nature Switzerland
DOI: 10.1007/978-3-032-08049-3_3

Evaluating Social Acceptance of eXtended Reality (XR) Agent Technology: A User Study

Autorzy: Quamara, Megha, Schmuck, Viktor, Iani, CristinaPrimavesi, Axel, Plaum, Alexander, Viganò, Luca
Wydawca: Zenodo
DOI: 10.5281/ZENODO.16882793

A Long-Range Mutual Gaze Detector for HRI

Autorzy: Simone Arreghini, Gabriele Abbate, Alessandro Giusti, Antonio Paolillo
Opublikowane w: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2025
Wydawca: ACM
DOI: 10.1145/3610977.3637470

A Mixed Reality Interface for Human-Swarm Interaction

Autorzy: Villani, Valeria,Capelli, Beatrice, Sabattini, Lorenzo
Wydawca: Zenodo
DOI: 10.5281/ZENODO.14246842

Peer reviewed articles (3)

TAG2G: A Diffusion-Based Approach to Interlocutor-Aware Co-Speech Gesture Generation

Autorzy: Filippo Favali, Viktor Schmuck, Valeria Villani, Oya Celiktutan
Opublikowane w: Electronics, Numer 13, 2025, ISSN 2079-9292
Wydawca: MDPI AG
DOI: 10.3390/ELECTRONICS13173364

Self-supervised prediction of the intention to interact with a service robot

Autorzy: Gabriele Abbate, Alessandro Giusti, Viktor Schmuck, Oya Celiktutan, Antonio Paolillo
Opublikowane w: Robotics and Autonomous Systems, Numer 171, 2025, ISSN 0921-8890
Wydawca: Elsevier BV
DOI: 10.1016/J.ROBOT.2023.104568

Imitation learning-based Direct Visual Servoing using the large projection formulation

Autorzy: Sayantan Auddy, Antonio Paolillo, Justus Piater, Matteo Saveriano
Opublikowane w: Robotics and Autonomous Systems, Numer 190, 2025, ISSN 0921-8890
Wydawca: Elsevier BV
DOI: 10.1016/J.ROBOT.2025.104971