Periodic Reporting for period 3 - ViAjeRo (ViAjeRo: Virtual and Augmented Reality passenger experiences)
Reporting period: 2022-09-01 to 2024-02-29
The aim of ViAjeRo (Traveller in Spanish) is to radically improve all passenger journeys by facilitating the use of immersive Virtual and Augmented Reality (together called XR) to support entertainment, work and collaboration when on the move.
In Europe, people travel an average of 12,000km per year on private and public transport, in cars, buses, planes and trains. These journeys are often repetitive and wasted time. This total will rise with the arrival of fully autonomous cars, which free drivers to become passengers. The potential to recover this lost time is impeded by 3 significant challenges:
Interaction in confined spaces: These limit interactivity, and force us to rely on small displays such as phones or seatback screens;
Social acceptability: We may share the space with others, inducing a pressure to conform, inhibiting technology use;
Motion sickness: Many people get sick when they read or play games in vehicles. Once experienced, it can take hours for symptoms to resolve.
VR/AR headsets could allow passengers to use their travel time in new, productive, exciting ways, but only if bold research is undertaken to overcome these fundamental challenges. ViAjeRo will use VR/AR to do adventurous multidisciplinary work, unlocking the untapped potential of passengers. They will be able to use large virtual displays for productivity; escape the physical confines of the vehicle and become immersed in virtual experiences; and communicate with distant others through new embodied forms of communication – all whilst travelling.
In Europe, people travel an average of 12,000km per year on private and public transport, in cars, buses, planes and trains. These journeys are often repetitive and wasted time. This total will rise with the arrival of fully autonomous cars, which free drivers to become passengers. The potential to recover this lost time is impeded by 3 significant challenges:
Interaction in confined spaces: These limit interactivity, and force us to rely on small displays such as phones or seatback screens;
Social acceptability: We may share the space with others, inducing a pressure to conform, inhibiting technology use;
Motion sickness: Many people get sick when they read or play games in vehicles. Once experienced, it can take hours for symptoms to resolve.
VR/AR headsets could allow passengers to use their travel time in new, productive, exciting ways, but only if bold research is undertaken to overcome these fundamental challenges. ViAjeRo will use VR/AR to do adventurous multidisciplinary work, unlocking the untapped potential of passengers. They will be able to use large virtual displays for productivity; escape the physical confines of the vehicle and become immersed in virtual experiences; and communicate with distant others through new embodied forms of communication – all whilst travelling.
The work undertaken so far has focused around our three core research challenges: interaction techniques for confined spaces, social acceptability, and motion sickness.
Interaction in confined spaces
We focused on understanding the design space of the different passenger environments, including cars, aeroplanes, buses and trains. As a first attempt to understand this, we simulated a VR passenger airplane environment to evaluate three different AR-driven virtual display configurations (Horizontal, Vertical, and Focus), exploring their usability, user preferences, and the underlying factors that influenced those preferences. We found that the perception of invading other's personal space significantly influenced preferred layouts in transit contexts. We found that the placement of virtual content was highly influenced by the physical environment, depending on the position of both passengers and physical obstacles. Results from our experiment showed: significant usage of the physical environment to align displays; strong social effects meant avoiding placing displays over other passengers or their belongings; and use of displays for shielding oneself from others. Our findings show the unique challenges posed by the mode of transport and presence of others on the use of AR for mobile productivity in the future.
Social acceptability:
XR headsets have the capability to create virtual content anywhere around the user, disconnecting users from their surroundings. To create solutions for more socially acceptable VR experiences, we first needed to understand what factors and concerns influence social acceptance of VR in different travelling contexts. We found that VR on shorter journeys was less socially acceptable because participants felt that there was insufficient time to enjoy the VR experiences and it was important to follow the route more closely, compared to longer journeys that had a clear final stop, allowing the user to immerse themselves in VR. We also found that accidentally injuring someone, losing one’s belongings or being unable to react if fellow passengers require attention were key barriers to VR adoption. We established that the lack of awareness of immediate objects, one's belongings and other passengers were seen as major safety and comfort concerns for VR use in transit. Therefore, our following work investigated how the visibility of these objects could influence user experience and social acceptance of VR. We designed a user study that brought these elements into Virtual Reality, testing our users’ reactions. We found that other passengers were the most important cue from reality that participants wanted to see, whilst inanimate objects were of less interest. This work sets initial guidelines for more acceptable VR experiences.
Motion sickness
Mitigating motion sickness using neurostimulation is a new era of the in-car use of VR, because it focuses on the enhancement or inhibition of human brain functions themselves without any requirement of sophisticated VR software development. We have a 3-step research plan for our neurostimulation study. Step 1 is to explore the brain biomarkers of VR motion sickness. Step 2 is to implement neurostimulation based on the discovered brain biomarkers, which will be followed by Step 3, an in-car study that aims to verify the developed neurostimulation approach. We have completed steps 1 and 2 and step 3 will begin shortly. For step 1, we confirmed that the left parietal region of our brain is associated with VR motion sickness by using conventional functional segregation methods. On top of this, we found a new brain biomarker (that is, frontoparietal coherence) that is associated with VR motion sickness. This finding not only confirms the involvement of the newly proposed human vestibular network in VR motion sickness but also lays the foundation of the implementation of transcranial alternating current stimulation (tACS), featuring simultaneously modulating multiple regions' brain states. We also found that VR motion sickness impaired cognitive control ability represented by the degree of attentional engagement. Step 2 is now complete. We found that the number of participants who withdrew from the experiments in the non-tACS group (that is, the control group) was three times that of the tACS group.
Interaction in confined spaces
We focused on understanding the design space of the different passenger environments, including cars, aeroplanes, buses and trains. As a first attempt to understand this, we simulated a VR passenger airplane environment to evaluate three different AR-driven virtual display configurations (Horizontal, Vertical, and Focus), exploring their usability, user preferences, and the underlying factors that influenced those preferences. We found that the perception of invading other's personal space significantly influenced preferred layouts in transit contexts. We found that the placement of virtual content was highly influenced by the physical environment, depending on the position of both passengers and physical obstacles. Results from our experiment showed: significant usage of the physical environment to align displays; strong social effects meant avoiding placing displays over other passengers or their belongings; and use of displays for shielding oneself from others. Our findings show the unique challenges posed by the mode of transport and presence of others on the use of AR for mobile productivity in the future.
Social acceptability:
XR headsets have the capability to create virtual content anywhere around the user, disconnecting users from their surroundings. To create solutions for more socially acceptable VR experiences, we first needed to understand what factors and concerns influence social acceptance of VR in different travelling contexts. We found that VR on shorter journeys was less socially acceptable because participants felt that there was insufficient time to enjoy the VR experiences and it was important to follow the route more closely, compared to longer journeys that had a clear final stop, allowing the user to immerse themselves in VR. We also found that accidentally injuring someone, losing one’s belongings or being unable to react if fellow passengers require attention were key barriers to VR adoption. We established that the lack of awareness of immediate objects, one's belongings and other passengers were seen as major safety and comfort concerns for VR use in transit. Therefore, our following work investigated how the visibility of these objects could influence user experience and social acceptance of VR. We designed a user study that brought these elements into Virtual Reality, testing our users’ reactions. We found that other passengers were the most important cue from reality that participants wanted to see, whilst inanimate objects were of less interest. This work sets initial guidelines for more acceptable VR experiences.
Motion sickness
Mitigating motion sickness using neurostimulation is a new era of the in-car use of VR, because it focuses on the enhancement or inhibition of human brain functions themselves without any requirement of sophisticated VR software development. We have a 3-step research plan for our neurostimulation study. Step 1 is to explore the brain biomarkers of VR motion sickness. Step 2 is to implement neurostimulation based on the discovered brain biomarkers, which will be followed by Step 3, an in-car study that aims to verify the developed neurostimulation approach. We have completed steps 1 and 2 and step 3 will begin shortly. For step 1, we confirmed that the left parietal region of our brain is associated with VR motion sickness by using conventional functional segregation methods. On top of this, we found a new brain biomarker (that is, frontoparietal coherence) that is associated with VR motion sickness. This finding not only confirms the involvement of the newly proposed human vestibular network in VR motion sickness but also lays the foundation of the implementation of transcranial alternating current stimulation (tACS), featuring simultaneously modulating multiple regions' brain states. We also found that VR motion sickness impaired cognitive control ability represented by the degree of attentional engagement. Step 2 is now complete. We found that the number of participants who withdrew from the experiments in the non-tACS group (that is, the control group) was three times that of the tACS group.
We have made significant steps beyond the state of the art in our three focus areas: interaction techniques for confined spaces, social acceptability, and motion sickness.
Interaction techniques: as well as understanding the layout of virtual displays and their effects on social acceptance, we have also investigated the use of passive haptic surfaces and perceptual manipulations to increase comfort. This allows faster and more reliable input in confined seating scenarios. We have also investigated how the smaller physical movements that can be made in confined seating can be used to control large VR scences, for example using movement gain. We have also built a sensing platform that allows us to sense car motion in close to real time to enable VR use in cars and other forms of transport.
Social acceptability: we have develop the idea of 'reality anchors' which are the key aspects of reality (people, signage, belongings) that need to be brought in to immersive VR to make it acceptable to use in different forms of transport.
Motion sickness: alongside the work on neurostimulation, we have looked at how visual and audio manipulations can also reduce motion sickness. 3D audio along with appropriate visual cues can reduce motion sickness. Rotating the display window by a small amount in the direction of the physical movement can also reduce motion sickness
Interaction techniques: as well as understanding the layout of virtual displays and their effects on social acceptance, we have also investigated the use of passive haptic surfaces and perceptual manipulations to increase comfort. This allows faster and more reliable input in confined seating scenarios. We have also investigated how the smaller physical movements that can be made in confined seating can be used to control large VR scences, for example using movement gain. We have also built a sensing platform that allows us to sense car motion in close to real time to enable VR use in cars and other forms of transport.
Social acceptability: we have develop the idea of 'reality anchors' which are the key aspects of reality (people, signage, belongings) that need to be brought in to immersive VR to make it acceptable to use in different forms of transport.
Motion sickness: alongside the work on neurostimulation, we have looked at how visual and audio manipulations can also reduce motion sickness. 3D audio along with appropriate visual cues can reduce motion sickness. Rotating the display window by a small amount in the direction of the physical movement can also reduce motion sickness