Extended Reality (XR) is an umbrella term for advanced interactive systems such as Virtual and Augmented Reality systems. XR systems are emerging in different areas such as entertainment (e.g. games), training (e.g. soldiers, healthcare professionals) and retail (e.g. Rolex, IKEA), amongst others. Testing these complex systems is critical to ensure that they function correctly and deliver a high-quality user experience (UX). As the complexity of these systems keeps increasing, the XR industry now finds itself confronting a soaring engineering challenge: XR's fine-grained and high level of interactivity and realism make such systems very hard and expensive to test.
The current XR systems authoring and development toolset poses no XR testing technology beyond rudimentary record and replay tools with very little automation support. They typically do not support comprehensive test scenarios, and more importantly, the test cases are not easily replicable and do not accommodate changes to the system, making them quickly obsolete. This lack of suitable testing tools means that testing complex XR systems involves hours of manual work done by human testers, increasing costs and difficulty associated with testing the systems. The primary issue associated with these testing difficulties relies mainly on time. Given the time it takes to test one version of one XR system effectively, this lack of an appropriate toolset actively hinders the industry's growth and ability to respond to market demands for sophisticated systems that provide a high-quality user experience.
This project aims to build novel verification and validation technology and a toolkit for XR systems based on techniques from AI to provide learning and reasoning over a virtual world. With this technology, XR developers can deploy powerful test agents to automatically explore and test the correct parameters of their virtual worlds as they iteratively develop and refine them.
The agents will be driven by testing goals that developers may define and configure. These goals include the exploration of parameters and concerns with testing coverage. The agents will have diverse capabilities which can be achieved with the integration of external tools (e.g. FAtiMA toolkit - https://fatima-toolkit.eu/ - and TESTAR -
https://testar.org(se abrirá en una nueva ventana)).
The agents will be able to test a wide range of functionalities and address UX issues, an essential part of XR systems. We aim to develop socio-emotional and functional test agents to improve the validation and verification of XR systems, making them more systematic and cost-effective. Our main output will be a framework that allows developers and testers to formulate test goals and deploy intelligent agents to interact with the system.
Intelligent agents can bring several advantages to automated testing, from "human-like" play-testing to the creation of personas, ensuring XR systems are adapted to all relevant demographics. Several agents can also be employed to test systems that can be used by multiple users simultaneously.
The iv4XR project aims to deploy the developed Framework to three use-cases that will provide continuous benchmarking for the test agents developed:
an advanced training simulation system from THALES,
an advanced 3D game called Space Engineers from GoodAI, and
an advanced structure/building monitoring system from Gameware Europe.
All three use cases use different interaction and visualisation technologies, each representing a unique virtual world, reflecting the diversity we have to deal with in the XR market.
