Periodic Reporting for period 2 - BRIDGES (A hyBRID (physical-diGital) multi-user Extended reality platform as a stimulus for industry uptake of interactive technologieS)
Reporting period: 2021-12-01 to 2023-03-31
Immersive interactive technologies have the potential to build on Europe’s creativity, skills and cultural diversity, and impact various domains – from manufacturing, engineering and architecture to education, healthcare, the arts, entertainment and culture. Virtual Reality (VR), particularly, has come of age, now evolving, along with Augmented Reality (AR) and Mixed Reality (MR), into a body of immersive technologies that come together under the all-encompassing notion of eXtended Reality (XR).
Thanks to the development of more comfortable, affordable and easier-to-use hardware, as well as the design of applications to support and leverage the advantages of this hardware, the once nascent XR technologies have now reached a tipping point for large scale adoption. Nevertheless, despite the advancements the application and adoption still suffers from a number of shortcomings like the separation of digital-virtual from the physical world, and limited collaboration between people. To address these shortcomings, the BRIDGES innovation action builds upon the product of collaboration between research and industry, expanding and augmenting it to deliver a flexible and scalable solution that can be applied to a variety of situations.
BRIDGES aims to support and boost the pan-European interactive technologies industry through the development of a holistic solution for (remote and co-located) group interaction in room-scale immersive mixed reality (or eXtended Reality - XR) environments that blend the physical and virtual space.
The BRIDGES solution is based on an existing platform, the Immersive Deck®, which has been researched and developed by consortium members TUW and IW. Within BRIDGES we are enhancing technical and operational aspects, piloting and testing it extensively in various domains to validate in real-world settings of major international airports in Germany and Greece for training firefighters and first responders, and in a highly visited cultural center in Greece, for enhancing the learning and recreational experience of visitors.
The extensive validation effort is expected to lead to the short-term commercial deployment of the BRIDGES end-results in the specific domains addressed by the project pilots, but also to provide replicability recommendations and scale-up guidelines for deploying the resulting XR platform in other industrial and entertainment domains in the longer term.
Overall Objectives of the project include:
· a set of custom and commercial off-the-shelf technologies, assembled into a complete XR platform installed at industrial and cultural settings for training and informal learning.
· a user-centered methodology and digital toolkit for creating and delivering multi-participant - sensorial XR experiences.
· a set of XR experiences, created to showcase the possibilities of the BRIDGES solution in training for fire fighters and visiting an ancient home
· an evaluation framework, produced after extensive testing to assess the effectiveness of the BRIDGES approach, and of XR at large.
Thanks to the development of more comfortable, affordable and easier-to-use hardware, as well as the design of applications to support and leverage the advantages of this hardware, the once nascent XR technologies have now reached a tipping point for large scale adoption. Nevertheless, despite the advancements the application and adoption still suffers from a number of shortcomings like the separation of digital-virtual from the physical world, and limited collaboration between people. To address these shortcomings, the BRIDGES innovation action builds upon the product of collaboration between research and industry, expanding and augmenting it to deliver a flexible and scalable solution that can be applied to a variety of situations.
BRIDGES aims to support and boost the pan-European interactive technologies industry through the development of a holistic solution for (remote and co-located) group interaction in room-scale immersive mixed reality (or eXtended Reality - XR) environments that blend the physical and virtual space.
The BRIDGES solution is based on an existing platform, the Immersive Deck®, which has been researched and developed by consortium members TUW and IW. Within BRIDGES we are enhancing technical and operational aspects, piloting and testing it extensively in various domains to validate in real-world settings of major international airports in Germany and Greece for training firefighters and first responders, and in a highly visited cultural center in Greece, for enhancing the learning and recreational experience of visitors.
The extensive validation effort is expected to lead to the short-term commercial deployment of the BRIDGES end-results in the specific domains addressed by the project pilots, but also to provide replicability recommendations and scale-up guidelines for deploying the resulting XR platform in other industrial and entertainment domains in the longer term.
Overall Objectives of the project include:
· a set of custom and commercial off-the-shelf technologies, assembled into a complete XR platform installed at industrial and cultural settings for training and informal learning.
· a user-centered methodology and digital toolkit for creating and delivering multi-participant - sensorial XR experiences.
· a set of XR experiences, created to showcase the possibilities of the BRIDGES solution in training for fire fighters and visiting an ancient home
· an evaluation framework, produced after extensive testing to assess the effectiveness of the BRIDGES approach, and of XR at large.
The project has managed to realise a user-centred experience design philosophy and a platform architecture that contains innovative components. The performed work includes:
- identifying user requirements via workshops and creating experience design & scenarios for the pilots
- Realise a platform architecture, that is replicable, with several enhancements on top of the pre-existing Immersive Deck®, including an improved Mapping Process (a new robot was created - not part of the initial proposal), advances in tracking, mixed reality and multi-sensory output modules and networking module and multi-user support.
- Establish effective cooperation with end-users in order to prepare and work towards pilots implementation. Venue selection included selection of proper rooms and installation plans to host the:
o Pilot 1 scenario dealing with industrial training - Firefighters training for Airports Berlin and Athens
o Establishing cooperation with fire brigades to properly design training scenarios and content based on real training requirements. Fire Brigades operate under strict rules and face dangerous situations; skills training potential via XR was showcased and met with enthusiasm in terms of offered functionalities, safety but also cost reduction for dangerous environment skills training
o The Greek Fire Brigade signed a cooperation agreement with the project and participates in the Advisory User Group (AUG), along with the German based Aircraft Rescue and Firefighting (ARFF) organisation
o Pilot 2 scenario dealing with informal learning, an immersive cultural experience, allowing visitors of the venue to explore the life in an ancient Athenian home.
o Establishing cooperation with archaeologists and educational experts to properly design training scenarios and content based on user requirements. The Education Department of the world renown Museum of Cycladic Art joined the AUG.
o For both pilots design and customize the platform and of course procure the necessary equipment in order to create the on-site experience to be demonstrated in the next months.
-Establish data collection and management procedures, Establish market watch procedures, and Disseminating the results in conferences and social media
- identifying user requirements via workshops and creating experience design & scenarios for the pilots
- Realise a platform architecture, that is replicable, with several enhancements on top of the pre-existing Immersive Deck®, including an improved Mapping Process (a new robot was created - not part of the initial proposal), advances in tracking, mixed reality and multi-sensory output modules and networking module and multi-user support.
- Establish effective cooperation with end-users in order to prepare and work towards pilots implementation. Venue selection included selection of proper rooms and installation plans to host the:
o Pilot 1 scenario dealing with industrial training - Firefighters training for Airports Berlin and Athens
o Establishing cooperation with fire brigades to properly design training scenarios and content based on real training requirements. Fire Brigades operate under strict rules and face dangerous situations; skills training potential via XR was showcased and met with enthusiasm in terms of offered functionalities, safety but also cost reduction for dangerous environment skills training
o The Greek Fire Brigade signed a cooperation agreement with the project and participates in the Advisory User Group (AUG), along with the German based Aircraft Rescue and Firefighting (ARFF) organisation
o Pilot 2 scenario dealing with informal learning, an immersive cultural experience, allowing visitors of the venue to explore the life in an ancient Athenian home.
o Establishing cooperation with archaeologists and educational experts to properly design training scenarios and content based on user requirements. The Education Department of the world renown Museum of Cycladic Art joined the AUG.
o For both pilots design and customize the platform and of course procure the necessary equipment in order to create the on-site experience to be demonstrated in the next months.
-Establish data collection and management procedures, Establish market watch procedures, and Disseminating the results in conferences and social media
Several aspects of XR tech have evolved during the project including:
- Mapping process: Mapping is the most expensive and time-intensive part of using XR. The consortium created an automated robotic solution that makes it easier, cheaper and faster to adopt extended reality technology. This represents an extra outcome of this innovation action. As a result, the workflow has improved dramatically and delivers us a unique precision in comparison to other marker-based inside-out tracking systems. With these installation improvements we can now also address the market segment of fairs, exhibitions, conferences, and any short-term event. This is a unique selling preposition in the market for XR location-based platforms
- Tracking, Mixed Reality and Multi-Sensory Output Modules and Object tracking have been significantly enhanced and real equipment objects are accurately and efficiently visualized along with virtual content for VR-headsets
- Networking Module and Multi-User Support: The number of simultaneous users is one of the key factors especially in entertainment and educational sector. It raises the productivity of the Immersive Deck in comparison to other technical solutions and also supports the strength of the Immersive Deck technology to enable large deck scales by lowest cost per square meter
- Platform Architecture, Continuous Platform Integration and Optimization - Immersive Creator: This product is a SDK with an API for Unity3D, one of the most common graphic engines for VR applications in the world. This SDK allows the majority of VR developers to use standard programming codes for content integration into the Immersive Deck.
- Experience Design & Pilot Scenarios: XR-training experience for airport fire brigades offers a unique opportunity to hazard one of the most dangerous situations at modern airports. The high immersion and flexible scenario design allows trainer and trainees to experience missions inside aircrafts in different critical situations. The informal learning scenario provides an immersive cultural experience allowing a user group to walk inside and experience an ancient Athenian home.
- Mapping process: Mapping is the most expensive and time-intensive part of using XR. The consortium created an automated robotic solution that makes it easier, cheaper and faster to adopt extended reality technology. This represents an extra outcome of this innovation action. As a result, the workflow has improved dramatically and delivers us a unique precision in comparison to other marker-based inside-out tracking systems. With these installation improvements we can now also address the market segment of fairs, exhibitions, conferences, and any short-term event. This is a unique selling preposition in the market for XR location-based platforms
- Tracking, Mixed Reality and Multi-Sensory Output Modules and Object tracking have been significantly enhanced and real equipment objects are accurately and efficiently visualized along with virtual content for VR-headsets
- Networking Module and Multi-User Support: The number of simultaneous users is one of the key factors especially in entertainment and educational sector. It raises the productivity of the Immersive Deck in comparison to other technical solutions and also supports the strength of the Immersive Deck technology to enable large deck scales by lowest cost per square meter
- Platform Architecture, Continuous Platform Integration and Optimization - Immersive Creator: This product is a SDK with an API for Unity3D, one of the most common graphic engines for VR applications in the world. This SDK allows the majority of VR developers to use standard programming codes for content integration into the Immersive Deck.
- Experience Design & Pilot Scenarios: XR-training experience for airport fire brigades offers a unique opportunity to hazard one of the most dangerous situations at modern airports. The high immersion and flexible scenario design allows trainer and trainees to experience missions inside aircrafts in different critical situations. The informal learning scenario provides an immersive cultural experience allowing a user group to walk inside and experience an ancient Athenian home.