A unifying framework that integrates conceptual, computational, and experimental approaches to characterize social interactions in SHS has been developed. This framework is driven by the fundamental insight that human movement inherently carries information. The core hypothesis is that by leveraging principles of amplification and synchronization, we can modulate how this information is transmitted during social interactions in XR environments.
A new approach to IMU-based body tracking called "Joint Tracker”. We integrated the IMU with visual upper body tracking to limit the number of additional sensors. Calibration is limited to a single step and methods for auto-calibration have been investigated. A novel approach for 3D metric hand tracking has been developed and tested. The AI-based cognitive architectures (CAs) for virtual humans has been developed to improve motor coordination in a group. The communication platform architecture for Sharespace, designed for real-world use and built on Pixel Streaming ash been integrated into Enterprise Rainbow environment. In addition, a streaming animation system has been developed to allow for the animation of virtual characters and audio with lip animation has been implemented. The multi-focal near-eye augmented reality display has been further developed to mechanically incorporate a proprietary ultra-high refresh rate eye-tracking sub system. The first hardware integration has been completed and both systems have been mated.
The significant groundwork has been laid in relation to the ethical and socio-cultural evaluation of SHS. An ethical framework that directly speaks to the influence of these three dimensions (demographics, psychological and sociocultural) on SHS experiences is in development in the form of a series of policy briefs.
The three pilots have been planned. The first trial with experts in the Health scenario took place at the Barcelona Hospital in June 2024 and was successfully completed. The results will be presented at the 2024 World Pain Conference in Amsterdam on August 7, 2024. The Sports demonstrator is ready and will be shown at the Club France Olympic Games from 26 July to 13 August 2024. For the Art scenario, the artists have been recruited through an open call and have developed concepts for Sharespace performances in the context of the Ars Eletronica Festival from 5 to 8 September 2024.
