Turning extended reality into a space for social good
Extended reality (XR), which includes virtual reality and augmented reality, is a technology that merges the physical and virtual worlds. It provides immersive social and collaborative environments where people can meet, discuss and interact. While the technology offers new opportunities, it also brings important challenges. “Social interactions in XR can be difficult to manage, especially when discussions become complex, conflictive or unbalanced,” explains GuestXR(opens in new window) scientific coordinator Mel Slater, from the University of Barcelona(opens in new window). “There is also a risk of exclusion, unequal participation and communication barriers, particularly for people with hearing difficulties,” adds project member Amir Amedi from Reichman University(opens in new window).
Social psychology, neuroscience and behavioural science
The EU-funded GuestXR project, coordinated by Eurecat, set out to address these issues through supporting more constructive, inclusive and effective interactions in virtual environments. More specifically, it aimed to help detect and mitigate conflict situations, reduce antisocial behaviours, and ensure that all participants can take part in a more balanced way. “The core concept behind GuestXR is an artificial intelligence agent called the Guest, designed to operate within extended reality environments,” says Slater. “This agent is embedded directly into virtual or augmented spaces where people interact, observing both individual and group behaviour in real time.” Based on models from social psychology, neuroscience and behavioural science, the agent is able to interpret what is happening in the interaction and intervene when needed. Slater explains that its role is not to control the conversation, but to facilitate it. “For example, it can encourage quieter participants to contribute to the discussion, help maintain a constructive tone, reduce tension if a conflict arises, and guide the participants of the group to achieve the meeting’s objectives,” he notes.
XR technology in four use cases
The project team applied their XR technology to four use cases, including social situations that could trigger conflict or debate. The XR environment was designed to encourage participants to explore different perspectives in potentially conflictual situations. Historical protests were also simulated to help the team explore individual and group responses, and to assess the extent to which the Guest was able to influence participant engagement. The project team also created an innovative XR game in which players’ actions in cryptocurrency mining reflect the environmental impacts of unsustainable resource usage. The game confronts players with the consequences of resource scarcity, to raise environmental awareness, work led by Andrzej Nowak from the University of Warsaw(opens in new window). Finally, the project explored how to involve people with hearing impairments in XR environments. “New tools to optimise acoustics and enhance speech understanding in extended reality were developed, helping people with hearing difficulties to join and participate in meetings,” explains Umut Sayin, researcher at Eurecat.
Responsible innovation in XR
Following positive results, next steps include moving from experimental validation to more applied scenarios. “GuestXR technologies have already shown potential in areas such as moderated virtual meetings, education, training and social inclusion,” says Slater. “The climate-related XR application has been used in collaboration with educational organisations, and the protest simulation framework is being adapted for other types of scenarios.” Over the long term, the GuestXR project hopes to have demonstrated how XR can contribute towards a ‘metaverse for the good’, to be used as a space for positive and constructive social interaction. Through responsible innovation in XR, we have shown how artificial intelligence can be integrated into social environments in a way that supports human interaction while respecting ethical principles.