Embodied Social Experiences in Hybrid Shared Spaces

Projektinformationen

SHARESPACE

ID Finanzhilfevereinbarung: 101092889

DOI

10.3030/101092889

Projekt abgeschlossen

EK-Unterschriftsdatum 14 November 2022

Startdatum 1 Januar 2023

Enddatum 31 Dezember 2025

Finanziert unter

• Digital, Industry and Space

Gesamtkosten

€ 5 947 573,50

EU-Beitrag

€ 5 947 573,50

Koordiniert durch

DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH
Germany

CORDIS bietet Links zu öffentlichen Ergebnissen und Veröffentlichungen von HORIZONT-Projekten.

Links zu Ergebnissen und Veröffentlichungen von RP7-Projekten sowie Links zu einigen Typen spezifischer Ergebnisse wie Datensätzen und Software werden dynamisch von OpenAIRE abgerufen.

Leistungen

Demonstrators, pilots, prototypes (19)

Implemented integrated SHARESPACE prototypes v1

The deliverable consists of the first integrated prototype demonstration of SHS-system which will be used in the trial.

XR communication platform design and validation v2

The deliverable presents the second and final version of the XR-communication platform which will be used for the trials.

Avatar adaptation v1

The deliverable consists of the first implementation demonstrating the capability of the avatar to adapt to the local context, meaning avatar-to-environment interactions and social engagement with multiple humans/avatars.

Accurate Hand Kinematics Predictions from ego-vision v2

The deliverable presents the second and final version of accurate hand tracking and reconstruction from an ego-perspective.

Accurate Hand Kinematics Predictions from ego-vision v1

The deliverable presents the first implementation of hand tracking and reconstruction from an ego-perspective.

XR communication platform design and validation v1

The deliverable presents the XR-communication platform which will be used for the trials.

Self-calibrating ego-centric visual-intertial bofy tracking v1

The deliverable presents succinctly the first implementation of the self-calibration capability of the visual-inertial ego-centric body sensor network.

Multi-sensors and Multifocal XR display v1

The deliverable consists of the demonstrator with the first integration of the cameras and sensor system in the head-mounted display. The demonstrator is presented in a document, which is submitted as a deliverable.

Avatar animation v2

The deliverable describes the final definition of the virtual avatar used in SHS. It represents the updated version of D4.1.

Multi-sensors and Multifocal XR display v2

The deliverable consists of the updated and final version of prototype D3.7.

Spherical RGB and ToF scene capturing v1

This demonstrator consists of an evaluation of the 3D capability of the spherical ToF-camera for scanning a room or large space. The results are described succinctly in a document submitted as deliverable.

Style Preserving animation of avatars v1

This deliverable consists of a demonstrator presenting the animated avatar with a certain degree of autonomy. The motion styles are preserved and do violate the sensorimotor primitives.

Self-calibrating ego-centric visual-intertial bofy tracking v2

The deliverable presents and describes the updated and final version of D3.1.

Avatar animation v1

This deliverable consists of a demonstrator presenting the capability of the avatar animation. The results will be succinctly documented in a report.

Spherical RGB and ToF scene capturing v2

This demonstrator consists of the second version of the spherical ToF-camera for scanning a room or large space. The deliverable is an update of D3.5.

Style Preserving animation of avatars v2

The deliverable demonstrates the final capability of style preservation. it is the updated and final version of D4.5.

Photo-realistic scene neutral rendering v1

The report presents the first results on neural rendering with spherical images for large spaces.

Implemented integrated SHARESPACE prototypes v2

The deliverable consists of the second and final integrated prototype demonstration of SHS-system which will be used for the final validation and trials.

Photo-realistic scene neutral rendering v2

The report presents the final results on neural rendering with spherical images for large spaces. It is an updated version of D4.10.

Documents, reports (22)

Ethical framework v2

This report presents the ethical approach and principles of the SHS project, the constitution of the Advisory Board and its activities for the second period of the project, i.e. from the month 12 to 36.

Interim impact and communication assessment report phase 1

The deliverable presents the first assessment of the communication measures. The impact is evaluated through measurable metrics and will lead to an adjustment of the communication strategy.

Results on real-world scenario validation phase 1

This deliverable presents the results of the evaluation for all scenarios of phase 1 of the project.

SHARESPACE final workshop

The report presents a summary of the final SHARESPACE workshop and assesses its success and outreach.

Architecture design and validation (L3 avatars) v1

The deliverable describes the first approaches and results of the capability of the avatars L3.

Research Requirements for challenges and scenarios v1

This deliverable will present the results of task 1.3 about the use-case description and requirement definition at the project beginning. It will allow for the development of the system architecture.

Architecture design and validation (L1 and L2 avatars) v3

The deliverable represents the updated version of D5.2 and reports on the final architecture capabilities.

SHARESPACE avatar definition v1

The deliverable describes the virtual avatar from a technical point of view. This includes the definition of the animation and rendering features as well as API for integration.

Social interaction dataset v1

The deliverable describes the created dataset on social interaction. This includes in particular the scenarios/protocols and data format.

Detailed planning on scenario validation v1

The deliverable presents a detailed planning of the trial as conceived in task 6.1. It addresses all aspects of the trial preparation and proposed a detailed timeline with corresponding milestones.

Architecture design and validation (L1 and L2 avatars) v2

The deliverable represents an updated version of D5.1 and reports on the new architecture capabilities.

The SHARESPACE unified and operational framework

The deliverables proposes a synthetic view of existing approaches of embodied social interactions, in natural and in SHS spaces, including concepts of neuroscience, embodied cognition and social XR, Health. It will also maintain and update a Living Glossary about those concepts and terms over the project lifespan.

SHARESPACE open science initiatives

The deliverable is a report on the open-science event. It provides all information about the event, including the program, number of participants, and outreach.

Ethical Framework v1

This report presents the ethical approach and principles of the SHS project, the constitution of the Advisory Board and its activities within the first 12 months.

Final impact and communication assessment report phase 2

The deliverable presents the final assessment of the communication measures. The impact is evaluated through accurate metrics and provide an overview of the success of the communication strategy.

Social interaction dataset v2

This deliverable describes the final dataset on social interaction delivered at the end of the project. It is an updated version of D2.11.

Architecture design and validation (L3 avatars) v2

The deliverable describes the final approach and results of the capability of the avatars L3. It represents the updated and final version of report D5.4.

Results on ethical scenarios validation phase 1

The deliverable provides an ethical evaluation of the trial scenarios and documents the results of the task 6.4, phase 1.

Architecture design and validation (L1 and L2 avatars) v1

The deliverable presents the results of the first architecture for L1 and L2 avatars, which generate reference motions for the avatar by considering the metrics to be minimised.

Detailed planning on scenario validation v2

The deliverable presents a detailed planning of the second trial. It addresses all aspects of the trial preparation and proposed a detailed timeline with corresponding milestones. It is a second version of the D6.1.

Results on ethical scenarios validation phase 2

The deliverable provides an ethical evaluation of the trial scenarios and documents the results of the task 6.4, phase 2.

Results on real-world scenario validation phase 2

This deliverable presents the result of the evaluation for all scenarios of phase 2 of the project.

Veröffentlichungen

Peer reviewed articles (7)

Distributed control for geometric pattern formation of large-scale multirobot systems

Autoren: Andrea Giusti, Gian Carlo Maffettone, Davide Fiore, Marco Coraggio, Mario di Bernardo
Veröffentlicht in: Frontiers in Robotics and AI, Ausgabe 10-2023, 2023, ISSN 1664-3224
Herausgeber: Frontiers
DOI: 10.48550/arXiv.2207.14567

Kinematic priming of action predictions

Autoren: Eugenio Scaliti, Kiri Pullar, Giulia Borghini, Andrea Cavallo, Stefano Panzeri, Cristina Becchio
Veröffentlicht in: CellPress, Ausgabe 33-13, 2023, ISSN 0092-8674
Herausgeber: Cell Press
DOI: 10.1016/j.cub.2023.05.055

The emerging role of virtual reality as an adjunct to procedural sedation and anesthesia

Autoren: Rita Hitching; Hunter G. Hoffman; Azucena Garcia-Palacios; Maheen M. Adamson; Esmeralda Madrigal; Wadee Alhalabi; Ahad Alhudali; Mariana Sampaio; Barry Peterson; Miles R. Fontenot; Keira P. Mason
Veröffentlicht in: Journal of Clinical Medicine, Ausgabe 12-3, 2023, ISSN 2077-0383
Herausgeber: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/jcm12030843

Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning

Autoren: De Lellis, Francesco; Coraggio, Marco; Russo, Giovanni; Musolesi, Mirco; di Bernardo, Mario
Veröffentlicht in: IEEE, 2023, ISSN 1803-7232
Herausgeber: IEEE
DOI: 10.48550/arxiv.2311.10026

Local Convergence of Multi-Agent Systems Toward Rigid Lattices

Autoren: Andrea Giusti; Marco Coraggio; Mario di Bernardo
Veröffentlicht in: IEEE, 2023, ISSN 1803-7232
Herausgeber: IEEE
DOI: 10.1109/lcsys.2023.3289060

Data-driven architecture to encode information in the kinematics of robots and artificial avatars

Autoren: Francesco De Lellis, Marco Coraggio, Nathan C. Foster, Riccardo Villa, Cristina Becchio, Mario di Bernardo
Veröffentlicht in: IEEE, 2024, ISSN 1803-7232
Herausgeber: IEEE
DOI: 10.48550/arXiv.2403.06557

Action prediction in psychosis

Autoren: Noemi Montobio
Veröffentlicht in: SpringerNature, Ausgabe 1264, 2024, ISSN 2193-1801
Herausgeber: Springer Science and Business Media Deutschland GmbH
DOI: 10.1038/s41537-023-00429-x.

Conference proceedings (11)

Multi - Modal Signal Processing for Avatar Motion Adaptation

Autoren: Amir Azizi, Panayiotis Charalambous, Yiorgos Chrysanthou
Veröffentlicht in: 2025 25th International Conference on Digital Signal Processing (DSP), 2025, ISSN 2165-3577
Herausgeber: IEEE
DOI: 10.1109/DSP65409.2025.11075085

1D-DiffNPHR: 1D Diffusion Neural Parametric Head Reconstruction using a single image.

Autoren: Jaiswal, P., Habtegebrial, T. A., & Stricker, D.
Veröffentlicht in: 2025
Herausgeber: ICPRAM

“Swinging with my heart”: Impact of an aversive stimulation on affective states, social bonding, cardiac activity and individual kinematics during group synchrony

Autoren: Ayache, J., Laroche, J., Pistre, C., Caraminot, O., Pla, S., Bieńkiewicz, M., & Bardy, B. G.
Veröffentlicht in: 2025
Herausgeber: IEEE

The Unfaithful Copy: Performed AI ‘Personhood’in Mixed Reality Platforms

Autoren: Bryson, K., Sutherland, J., & Richardson, K.
Veröffentlicht in: VTT Technology, Ausgabe 432, 2024, ISSN 2242-122X
Herausgeber: VTT
DOI: 10.32040/2242-122X.2024.T432

Rendering Togetherness: Embodied Social Synchronization in Multi-User VR

Autoren: Julia Ayache, Julien Laroche, Maxime Hérissé, Pierre Jean, Anna Katharina Hebborn, Didier Stricker, Benoit G. Bardy
Veröffentlicht in: 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2025, ISSN 2473-0726
Herausgeber: IEEE
DOI: 10.1109/ISMAR67309.2025.00079

Improving Image Reconstruction using Incremental PCA-Embedded Convolutional Variational Auto- Encoder

Autoren: Amir Azizi, Panayiotis Charalambous, Yiorgos Chrysanthou
Veröffentlicht in: WSCG, 2024
Herausgeber: WSCG
DOI: 10.24132/10.24132/CSRN.3401.12

Control- Tutored Deep Reinforcement Learning

Autoren: Francesco De Lellis, Marco Coraggio, Giovanni Russo, Mirco Musolesi, Mario di Bernardo
Veröffentlicht in: PMLR, Ausgabe 211, 2023, ISSN 2640-3498
Herausgeber: PMLR
DOI: 10.48550/arXiv.2212.01343

HI^2: Sparse-View 3D Object Reconstruction with a Hybrid Implicit Initialization.

Autoren: Jaiswal, P., & Stricker, D.
Veröffentlicht in: 2025
Herausgeber: ICPRAM

Demonstrating the effectiveness of combining heuristic and data-driven methods to achieve scalable and adaptive motion styles

Autoren: Amir Azizi, Panayiotis Charalambous, Yiorgos Chrysanthou
Veröffentlicht in: 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2025
Herausgeber: IEEE
DOI: 10.1109/VRW66409.2025.00144

Learning-based cognitive architecture for enhancing coordination in human groups

Autoren: Antonio Grotta, Marco Coraggio, Antonio Spallone, Francesco De Lellis, Mario di Bernardo
Veröffentlicht in: IFAC, 2024, ISSN 0000-1118
Herausgeber: IFAC
DOI: 10.48550/arXiv.2406.06297

Online Phase Estimation of Human Oscillatory Motion Using Deep Learning.

Autoren: Grotta, A. & De Lellis, F.
Veröffentlicht in: 2025
Herausgeber: ARXiv

Other (5)

Local convergence of multi-agent systems towards triangular patterns

Autoren: Giusti, Andrea; Coraggio, Marco; di Bernardo, Mario
Veröffentlicht in: IEEE, 2023, ISSN 1803-7232
Herausgeber: IEEE
DOI: 10.48550/arxiv.2303.11865

eXtended Reality of socio-motor interactions: Current Trends and Ethical Considerations for Mixed Reality Environments Design

Autoren: Julia Ayache, Marta Bienkiewicz, Kathleen Richardson, Benoit Bardy
Veröffentlicht in: ACM, 2023
Herausgeber: ACM
DOI: 10.1145/3610661.361798

Impact of group synchrony, postural stability and agency on cybersickness symptoms.

Autoren: Ayache, J., Pistre, C., Jean, P., Hebborn, A. K., Stricker, D., & Bardy, B. (2024, November)
Veröffentlicht in: 2024
Herausgeber: EuroXR

Data-driven design of complex network structures to promote synchronization

Autoren: Coraggio, Marco; di Bernardo, Mario
Veröffentlicht in: 2023
Herausgeber: N/A
DOI: 10.48550/arxiv.2309.10941

Continuification Control of Large-Scale Multiagent Systems Under Limited Sensing and Structural Perturbations

Autoren: Gian Carlo Maffettone; Maurizio Porfiri; Mario Di Bernardo
Veröffentlicht in: IEEE, 2023
Herausgeber: 1803-7232
DOI: 10.48550/arxiv.2303.13246

Book chapters (1)

Five Reality Types: The Embedded Ethicist in VR and Mixed-Reality Platforms

Autoren: Kathleen Bryson, Jessica Sutherland, Kathleen Richardson
Veröffentlicht in: Smart Innovation, Systems and Technologies, Recent Trends of AI Technologies and Virtual Reality, 2025, ISBN 978-981-96-1154-6
Herausgeber: Springer Nature Singapore
DOI: 10.1007/978-981-96-1154-6_9