Community Research and Development Information Service - CORDIS

H2020

AUGGMED Report Summary

Project ID: 653590
Funded under: H2020-EU.3.7.

Periodic Reporting for period 1 - AUGGMED (Automated Serious Game Scenario Generator for Mixed Reality Training)

Reporting period: 2015-06-01 to 2016-11-30

Summary of the context and overall objectives of the project

Terrorists and perpetrators of organised crime are continually inventing new and unpredictable methods to carry out attacks, which are increasingly unfamiliar to the security forces and therefore more difficult to predict, prepare and defend against. Currently security forces are trained using traditional methods such as live scenario training though full-scale field exercises. This type of training can be very costly and time-consuming in addition it can only offer a limited number of scenarios that individually require major resources to plan and carryout. This limits the amount and variety of situations for which security forces are currently trained and makes it difficult to maintain uniform levels of skill, judgement and preparedness across teams and individuals.
The aim of AUGGMED is to develop a serious gaming platform to train law enforcement agencies, first responders and operators of critical infrastructure in their response to terrorist and organised crime attacks.
AUGGMED is being developed to enable both single ‘player’ and team-based training. Training is aimed at first responders, law enforcement agencies and operators of critical infrastructure with different level of expertise from different organizations to better prepare them to respond to terrorist and organized crime threats. The AUGGMED platform will facilitate the generation of non-linear scenarios tailored to suit the needs of individual trainees with learning outcomes that will improve the acquisition of emotional management, analytical thinking, problem solving and decision making skills. The game scenarios will include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through a number of different methods including Virtual Reality (VR), immersive VR and Mixed Reality (MR) environments with multimodal interfaces.
AUGGMED will deliver highly realistic training scenarios and advanced interactivity while encouraging infrastructure security staff and first responders to engage and actively participate in the training process. The AUGGMED platform will also include tools for trainers, enabling them to set learning objectives, define scenarios, monitor training sessions, modify scenarios and provide feedback in real-time, as well as evaluate trainee performance and set training curricula for individual personnel in the post-training session phase.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

During the first half of the project, the focus has been on gathering the stakeholder requirements for the AUGGMED platform, developing the first version of the tool and testing this version with external stakeholders. Thus ensuring that the platform has potential use to fulfil training needs for law enforcement agencies, first responders and critical infrastructure providers for terrorist and serious crime attacks.

In the first five months of the project, the partners gathered expert opinion to define areas of vulnerability and potential threat scenarios. Experts were also involved in the definition of the current training procedures and requirements of the target end users. Partners used this information to define the measures by which the platform would be evaluated. Having established this baseline and evaluation criteria the partners started development of the platform specifications and then the build of the first version of the platform components. The automated Game Scenario Engine, VR environments and Trainer Tools were developed and a plan was established to test the system in the first mode pilot. This took place at WYP where the VR tool was demonstrated and tested by project partners and firearms officers. The platform was also later tested with external stakeholders of the project Advisory Board. Development has continued incorporating the feedback from the first pilot in preparation for pilot two of the immersive VR version of the platform and testing of the haptic vest.

In conjunction with the intensive development work, the partners have been involved in a number of dissemination events to promote the project and have begun planning strategies for exploiting the tool.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

There are several 3D simulation tools on the market that can be used for the training of soldiers, police officers or paramedical staff. While they are capable of rendering a convincing illusion of the real world, they all share one feature: complex processes like crowd behaviour or the spreading of fire and smoke are not (or only to a very limited degree) based on realistic, physical models and instead modelled ‘by hand’ to create a mere impression of realism. For a tool that is meant to prepare first responders for real-life disasters, this is a serious short-coming as their training is reliant not upon hard evidence but ultimately on an artist’s view of reality. AUGGMED addresses this issue by linking a 3D games engine to a state-of-the-art evacuation modelling tool (EXODUS), which in turn is using powerful fire simulation software (SMARTFIRE) to realistically model the evolution of a disaster scenario. Likewise, the effects of explosions (in particular the number of casualties, their spatial distribution and the severity of their injuries) is based on the best available evidence, thus providing users with the confidence that the rendered scenarios are as close to reality as is currently possible.
Moreover, the AUGGMED serious game will allow players to directly interact with crowd agents via hand gestures or spoken commands, thus allowing them to take control of a situation in a way that closely resembles a real-life scenario. In particular, players can interact with other players which enhances team work and/or collaboration with members of other organisations. The immersion of players into this type of realistically simulated environment will allow them to train their decision making skills and to experience (to a certain degree) emotional reactions that are inextricably linked to crisis scenarios.
In addition, AUGGMED will make use of a range of external devices (like VR headsets, walking platforms and haptic vests) to enhance the experience of immersion for the players.
As explained above, the AUGGMED serious game will be developed in three stages: the first mode will provide basic VR (based on conventional touchscreen devices), whereas the second and third mode will add HMDs and haptic vests to ultimately provide a fully immersive VR environment. The first mode has been tested in the first pilot; the feedback received from end users has been positive. Similarly, the progress of the work to date has been positively reviewed by the Advisory Board. Generally, the stakeholders pointed out the importance of an easy-to-use interface and of the realism of the training scenario; this includes a sufficient richness in the available details on which players base their decisions. For example, the AUGGMED partners are currently implementing the scenarios for the second pilot which involve suspicious parcels. Allowing users to inspect the parcel itself and its surrounding in detail can provide players with useful hints to decide how to act. This will enhance the training of decision making skills and hence maximise the impact of the system on the first responders’ training.
The ability to immerse oneself in a crisis scenario and hence to “experience” it is the main advantage of training simulation tools.

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top