GuestXR: A Machine Learning Agent for Social Harmony in eXtended Reality

Informacje na temat projektu

GuestXR

Identyfikator umowy o grant: 101017884

DOI

10.3030/101017884

Data podpisania przez KE 21 Grudnia 2021

Data rozpoczęcia 1 Stycznia 2022

Data zakończenia 31 Grudnia 2025

Finansowanie w ramach

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Koszt całkowity

€ 4 499 519,76

Wkład UE

€ 4 499 519,75

Koordynowany przez

FUNDACIO EURECAT
Spain

CORDIS oferuje możliwość skorzystania z odnośników do publicznie dostępnych publikacji i rezultatów projektów realizowanych w ramach programów ramowych HORYZONT.

Odnośniki do rezultatów i publikacji związanych z poszczególnymi projektami 7PR, a także odnośniki do niektórych konkretnych kategorii wyników, takich jak zbiory danych i oprogramowanie, są dynamicznie pobierane z systemu OpenAIRE .

Rezultaty

Documents, reports (8)

Ethics evaluation report I

Results of ethics evaluation, user-surveys and semi-structured interviews until Y2 (T5.6)

Public Launch of the project, and Communication, and Public Engagement Plans

It includes the public launch full visual identity set and full website t61

Survey and interview design

Intermediate result of Task 56

Initial Data Management Plan

Initial report of the project data management plan following the FAIR Data Management guidelines and template published by H2020 programme T73

Guidelines for ethics by design

Related to T14

Ethics State-of-the-Art and Literature Review

Intermediate result of task T56

First report: Use Case Applications

Consolidated intermediate results and progress in all use cases (T5.1, T5.2, T5.3, T5.4)

Dissemination and Exploitation Plans

Comprehensive plan to promote a unified message towards target audiences and the foreseen public engagement activities T61 T62 T63 T64 T65

Demonstrators, pilots, prototypes (3)

Public launch of Baseline Application

Intermedia result of T5.1

Multiple Representations of the Guest and its Actions Demonstration

Final results of T3.2

Real time representation of participants in a multi-person meeting - including facial expression representation - UB public demonstration

Results of task T3.6

Websites, patent fillings, videos etc. (2)

GuestXR Open Call

Launch of the GuestXR open call for small scale experiments and use of results in the project website (T6.3)

Project logo and webiste

Initial visual identity book and website landing page T61

Publikacje

Conference proceedings (14)

Motivational Interviewing Transcripts Annotated with Global Scores

Autorzy: Yosef, Stav; Zisquit, Moreah; Cohen, Ben; Brunstein, Anat Klomek; Bar, Kfik; Friedman, Doron
Opublikowane w: 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Numer 11, 2024
Wydawca: 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
DOI: 10.5281/zenodo.12792623

Persuasive Vibrations: Effects of Speech-Based Vibrations on Persuasion, Leadership, and Co-Presence During Verbal Communication in VR

Autorzy: Saint-Aubert, Justine; Argelaguet, Ferran; Macé, Marc J.-M.; Pacchierotti, Claudio; Amedi, Amir; Lécuyer, Anatole
Opublikowane w: VR 2023 - IEEE Conference on Virtual Reality and 3D User Interfaces, Numer 1, 2023
Wydawca: IEEE
DOI: 10.1109/vr55154.2023.00070

An objective evaluation of Hearing Aids and DNN-based speech enhancement in complex acoustic scenes

Autorzy: Gusó, Enric; Luberadzka, Joanna; Baig, Martí; Saraç, Umut Sayin; Serra, Xavier
Opublikowane w: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA23), Numer 1, 2023
Wydawca: IEEE
DOI: 10.48550/arxiv.2307.12888

Assessing Motivational Interviewing Sessions with AI-Generated Patient Simulations

Autorzy: Yosef, Stav; Zisquit, Moreah; Cohen, Ben; Brunstein, Anat Klomek; Bar, Kfir; Friedman, Doron
Opublikowane w: 9th Workshop on Computational Linguistics and Clinical Psychology, 2024
Wydawca: 9th Workshop on Computational Linguistics and Clinical Psychology
DOI: 10.5281/zenodo.12792592

"""Let’s ask what AI thinks then!"": Using LLMs for Collaborative Problem-Solving in Virtual Environments"

Autorzy: Esen Küçüktütüncü, Lisa Izzouzi
Opublikowane w: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2024, Strona(/y) 193-198
Wydawca: IEEE
DOI: 10.1109/ismar-adjunct64951.2024.00048

Haptics in the Metaverse: Haptic feedback for Virtual, Augmented, Mixed, and eXtended Realities

Autorzy: Pacchierotti, C., Chinello, F., Koumaditis, K., Di Luca, M., Ofek, E., & Georgiou, O.
Opublikowane w: IEEE Transactions on Haptics, 2024, ISSN 2329-4051
Wydawca: IEEE transactions on haptics
DOI: 10.5281/zenodo.10874975

Studying the influence of contact force on thermal perception at the fingertip

Autorzy: nilo Troisi, Jeanne Hecquard, Ferran Argelaguet, Justine Saint-Aubert,Marc J.-M. Macé, Anatole Lécuyer, Claudio Pacchierotti, Monica Malvezzi
Opublikowane w: Eurohaptics 2024, 2024
Wydawca: Eurohaptics
DOI: 10.5281/zenodo.11085942

Inceptor: An Open Source Tool for Automated Creation of 3D Social Scenarios

Autorzy: Dan Pollak; Jonathan Giron; Doron Friedman
Opublikowane w: IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2023, Strona(/y) pp. 474-476
Wydawca: IEEE
DOI: 10.1109/vrw58643.2023.00102

Cool Me Down: Effects of Thermal Feedback onCognitive Stress in Virtual Reality

Autorzy: Vincent Philippe, Jeanne Hecquard, Emilie Hummel, Ferran Argelaguet, Marc J.-M. Macé, Valérie Gouranton, Claudio Pacchierotti, Anatole Lécuyer, Justine Saint-Aubert
Opublikowane w: Eurohaptics 2024, 2024
Wydawca: EuropHaptics
DOI: 10.5281/zenodo.11085765

Sushi with Einstein: Enhancing Hybrid Live Events with LLM-Based Virtual Humans

Autorzy: Shoa, Alon and Oliva, Ramon and Slater, Mel and Friedman, Doron
Opublikowane w: IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents, 2023, Strona(/y) Pages 1 - 6
Wydawca: Association for Computing Machinery
DOI: 10.1145/3570945.3607317

Fostering empathy in social Virtual Reality through physiologically based affective haptic feedback

Autorzy: Hecquard, Jeanne; Saint-Aubert, Justine; Pacchierotti, Claudio; Argelaguet, Ferran; Lécuyer, Anatole; Macé, Marc
Opublikowane w: https://hal.science/hal-04099268, Numer 2, 2023, Strona(/y) pp. 78-84
Wydawca: IEEE
DOI: 10.1109/whc56415.2023.10224380

Breathing based immersive interactions for enhanced agency and body awareness: a claustrophobia motivated study

Autorzy: Iddo Yehoshua Wald; Amber Maimon; Lucas Keniger De Andrade Gensas; Noemi Guiot; Meshi Ben Oz; Benjamin W. Corn; Amir Amedi
Opublikowane w: CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, Strona(/y) Article No.: 59, Pages 1 - 7
Wydawca: Association for Computing Machinery
DOI: 10.1145/3544549.3585897

Comparison of synthesized Virtual Sound Environments with validated Hearing Aid experiments

Autorzy: Umut Sayin Saraç
Opublikowane w: Convention Express Paper 117, 2023
Wydawca: AES
DOI: 10.5281/zenodo.10076114

Harnessing Immersive Technologies: Modern Perspectives on Mental Health Support

Autorzy: Lisa Izzouzi, Esen Küçüktütüncü
Opublikowane w: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2024, Strona(/y) 295-298
Wydawca: IEEE
DOI: 10.1109/ismar-adjunct64951.2024.00067

Peer reviewed articles (20)

Interoceptive training with real‐time haptic versus visual heartbeat feedback

Autorzy: Olga Dobrushina, Yossi Tamim, Iddo Yehoshua Wald, Amber Maimon, Amir Amedi
Opublikowane w: Psychophysiology, Numer 61, 2024, ISSN 0048-5772
Wydawca: Blackwell Publishing Inc.
DOI: 10.1111/psyp.14648

Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device

Autorzy: Shira Shvadron; Shira Shvadron; Adi Snir; Adi Snir; Amber Maimon; Amber Maimon; Or Yizhar; Or Yizhar; Or Yizhar; Or Yizhar; Sapir Harel; Sapir Harel; Keinan Poradosu; Keinan Poradosu; Keinan Poradosu; Amir Amedi; Amir Amedi
Opublikowane w: Frontiers in Human Neuroscience, Numer 3, 2023, ISSN 1662-5161
Wydawca: Frontiers Research Foundation
DOI: 10.3389/fnhum.2023.1058617

Theta activity discriminates high-level, species-specific body processes

Autorzy: Chesley, J., Riecke, L., Lu, J., Vogels, R., & De Gelder, B.
Opublikowane w: Imaging Neuroscience, Numer 2, 2024, Strona(/y) 1–12, ISSN 2837-6056
Wydawca: MIT Press Direct
DOI: 10.1162/imag_a_00150

Differential contributions of body form, motion, and temporal information to subjective action understanding in naturalistic stimuli

Autorzy: Smekal Vojtěch , Poyo Solanas Marta , Fraats Evelyne I. C. , de Gelder Beatrice
Opublikowane w: Frontiers in Integrative Neuroscience, 2024, ISSN 1662-5145
Wydawca: Frontiers Research Foundation
DOI: 10.3389/fnint.2024.1302960

Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding

Autorzy: K. Cieśla; T. Wolak; A. Lorens; M. Mentzel; H. Skarżyński; Amir Amedi
Opublikowane w: Scientific reports, 2022, ISSN 2045-2322
Wydawca: Nature Publishing Group
DOI: 10.1038/s41598-022-06855-8

Modality-specific brain representations during automatic processing of face, voice and body expressions

Autorzy: Vaessen, M., Van der Heijden, K., & De Gelder, B
Opublikowane w: Frontiers in Neuroscience, Numer 17, 2024, ISSN 1662-453X
Wydawca: Frontiers
DOI: 10.3389/fnins.2023.1132088

The Making of a Newspaper Interview in Virtual Reality: Realistic Avatars, Philosophy, and Sushi

Autorzy: Oliva, Ramon and Beacco, Alejandro and Gallego, Jaime and Abellan, Raul Gallego and Slater, Mel
Opublikowane w: IEEE Computer Graphics and Applications, 2023, Strona(/y) pp. 117-125, ISSN 0272-1716
Wydawca: Institute of Electrical and Electronics Engineers
DOI: 10.1109/mcg.2023.3315761

Enhancing interoceptive sensibility through exteroceptive–interoceptive sensory substitution

Autorzy: Oran Goral, Iddo Yehoshua Wald, Amber Maimon, Adi Snir, Yulia Golland, Aviva Goral, Amir Amedi
Opublikowane w: Scientific Reports, Numer 14, 2024, ISSN 2045-2322
Wydawca: Nature Publishing Group
DOI: 10.1038/s41598-024-63231-4

A case study in phenomenology of visual experience with retinal prosthesis versus visual-to-auditory sensory substitution

Autorzy: Amber Maimon; Or Yizhar; Galit Buchs; Benedetta Heimler; Amir Amedi
Opublikowane w: Neuropsychologia, Numer 6, 2022, ISSN 0028-3932
Wydawca: Pergamon Press Ltd.
DOI: 10.1016/j.neuropsychologia.2022.108305

Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

Autorzy: Roni Arbel; Benedetta Heimler; Amir Amedi
Opublikowane w: Scientific Reports, Numer 12, 2022, ISSN 2045-2322
Wydawca: Nature Publishing Group
DOI: 10.1038/s41598-022-08187-z

The Topo-Speech sensory substitution system as a method of conveying spatial information to the blind and vision impaired

Autorzy: Amber Maimon; Amber Maimon; Iddo Yehoshua Wald; Iddo Yehoshua Wald; Meshi Ben Oz; Meshi Ben Oz; Sophie Codron; Sophie Codron; Ophir Netzer; Benedetta Heimler; Amir Amedi; Amir Amedi
Opublikowane w: Frontiers in Human Neuroscience, Vol 16 (2023), Numer 2, 2023, ISSN 1662-5161
Wydawca: Frontiers Research Foundation
DOI: 10.3389/fnhum.2022.1058093

Highly compromised auditory spatial perception in aided congenitally hearing-impaired and rapid improvement with tactile technology

Autorzy: Adi Snir, Katarzyna Cieśla, Rotem Vekslar, Amir Amedi
Opublikowane w: iScience, Numer 27, 2024, Strona(/y) 110808, ISSN 2589-0042
Wydawca: Elsevier Inc.
DOI: 10.1016/j.isci.2024.110808

Multi-sensory display of self-avatar's physiological state: virtual breathing and heart beating can increase sensation of effort in VR

Autorzy: Yann Moullec; Justine Saint-Aubert; Julien Manson; Melanie Cogne; Anatole Lecuyer
Opublikowane w: IEEE Transactions on Visualization and Computer Graphics, Numer 3, 2022, ISSN 1077-2626
Wydawca: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tvcg.2022.3203120

Localizing 3D motion through the fingertips: Following in the footsteps of elephants

Autorzy: Adi Snir, Katarzyna Cieśla, Gizem Ozdemir, Rotem Vekslar, Amir Amedi
Opublikowane w: iScience, Numer 27, 2024, Strona(/y) 109820, ISSN 2589-0042
Wydawca: Elsevier Inc.
DOI: 10.1016/j.isci.2024.109820

Early threat perception is independent of later cognitive and behavioral control. A virtual reality-EEG-ECG study

Autorzy: Lu, Juanzhi; Kemmerer, Selma K.; Riecke, Lars; de Gelder, Beatrice
Opublikowane w: Cerebral Cortex, Numer 2, 2023, Strona(/y) 8748-8758, ISSN 1460-2199
Wydawca: Cerebral Cortex
DOI: 10.1101/2023.02.22.529523

Ultrahigh Field fMRI Reveals Different Roles of the Temporal and Frontoparietal Cortices in Subjective Awareness

Autorzy: Poyo, M., Zhan, M., & De Gelder, B.
Opublikowane w: Journal of Neuroscience, Numer Vol 44, Numer 20, 2024, ISSN 1529-2401
Wydawca: Society for Neuroscience
DOI: 10.1523/jneurosci.0425-23.2023

Rapid plasticity in the ventral visual stream elicited by a newly learnt auditory script in congenitally blind adults

Autorzy: Roni, Arbel; Benedetta, Heimler; Amir, Amedi
Opublikowane w: Neuropsychologia, Numer 1, 2023, ISSN 0028-3932
Wydawca: Pergamon Press Ltd.
DOI: 10.1016/j.neuropsychologia.2023.108685

Testing geometry and 3D perception in children following vision restoring cataract-removal surgery

Autorzy: Amber Maimon; Amber Maimon; Ophir Netzer; Benedetta Heimler; Amir Amedi; Amir Amedi
Opublikowane w: Frontiers in Neuroscience, Vol 16 (2023), Numer 6, 2023, ISSN 1662-4548
Wydawca: Frontiers Research Foundation
DOI: 10.3389/fnins.2022.962817

Loss of action-related function and connectivity in the blind extrastriate body area

Autorzy: Or Yizhar; Or Yizhar; Or Yizhar; Zohar Tal; Amir Amedi; Amir Amedi
Opublikowane w: Frontiers in Neuroscience, Vol 17 (2023), Numer 1, 2023, ISSN 1662-4548
Wydawca: Frontiers Research Foundation
DOI: 10.3389/fnins.2023.973525

Activation of human visual area V6 during egocentric navigation with and without visual experience

Autorzy: Elena Aggius-Vella; Daniel-Robert Chebat; Shachar Maidenbaum; Amir Amedi
Opublikowane w: Current Biology, Numer 4, 2023, Strona(/y) Pages 1211-1219, ISSN 0960-9822
Wydawca: Cell Press
DOI: 10.1016/j.cub.2023.02.025