Final Report Summary - SAFECITI (Simulation Platform for the Analysis of Crowd Turmoil in Urban Environments with Training and Predictive Capabilities)
Executive Summary:
A fully functional platform has been developed according to the requisites exposed by the Spanish Police forces. The platform is designed for training purpose and integrates different modules. The main results are:
- Crowd simulation module. The system is able to simulate the behavior of crowds of up to 10.000 people.
- Urban Generation module. The system include the maps of real cities. The maps are generated with the information available of the street structure of the cities, so it is a realistic representation of the cities.
- User profiles. The system includes different interfaces and different functionalities for each profile of users (instructor or trainee)
- Adaptability. The system can be adapted to different security forces as it allows to edit important parameters according to the end users’ procedures.
- Communications. The platform includes the functionality of communication by chat and by voice, recreating realistic conditions of the operations.
- Predictive capability. The system recommends actions during an exercise based on previous similar situations
Dissemination efforts during this period were aimed at promoting the project in international events, in particular Milipol Paris. The platform has been shown to potential stakeholders related to the security industry. Other channels of dissemination have been the project website, procedures for internal communication, papers and scientific dissemination, presentations in conferences and events aimed at the security industries, and direct contacts with security forces and stakeholders. The project got the feedback from external participants through an Ethical Committee and an Advisory Board. Procedures to securely exchange confidential information have been agreed and respected. A Security Committee has controlled the dissemination.
Effective project management allowed the most cost-effective use of the EU contribution and produced the complete achievement of the results for the project.
Project Context and Objectives:
SAFECITI is a simulation platform that addresses the requirements of the civil security intelligence analysis community for the specific problem of crowd safety and anticipated threat control in cities. The platform is not only a “game” in the context of serious game technology, but an advanced simulator that aims to predict the complex behaviour of the crowd and to provide the analysts with new technology to improve the tactics and operations that are used in real situations.
This platform is designed both as a simulation platform for training purposes and as a predictive tool for operational use. The platform also includes measurement tools to evaluate the analysts based on their results. All the simulations will be stored in a database that will work as an historic repository, but more interestingly, as a large set of useful data that will be re-analysed by artificial intelligence algorithms to create predictive models. This predictive model will create a virtual analyst that is able to recommend actions based on the enriched analysis of hundreds or thousands of simulations.
The development of the Simulation Platform involves several technological challenges:
- Interactive graphical platform and interfaces and control system: development of the main control system, integration of all the modules in the interactive platform; user interfaces; scene setup; interactive workflow and control and network modules.
- Graphics Components: 3D models for human representation, vehicles and city infrastructure
- Generation of urban environment: development of an urban generation module to generate real city layouts based on real GIS data sources, e.g. from Madrid.
- Advanced crowd simulation: design and development of the crowd simulation engine that realistically simulates crowds of virtual humans, their behaviours and their actions on the environment.
- Database module and predictive model: development of database components and predictive models.
The main objectives of the project are:
• Intelligence analysis enhancement. Strengthening the performance and efficiency of intelligence analysts by means of new technologies.
• Simulation of people behaviour. The realistic simulation of crowd behaviours is one of the central aspects of the SAFECITI project.
• Interactive serious game platform. The crowd simulation engine will be integrated in the interactive serious game platform, and will be controlled using a specifically designed user interface.
• Predictive engine. The development of a predictive model will help the intelligence analysts in the decision-making processes.
Project Results:
The SAFECITI (Safe Citizen) Project was a two year project from 2014 to 2016, joining the efforts of six partners from academic to purely practical fields from different countries in the EU. The goal of the SAFECITI project was to develop a real-time application able of simulating realistic crowd behaviour in response to police actions during a crowd control scenario. Presented as an asymmetric online serious game, were an Instructor is able to control all the crowds behaviours in a scenario and several Trainees with an assigned set of Police officers and equipment have to deal with the situations generated by the Instructor and take the correct tactical decisions.
Entities simulating the police units are able to be commanded at high level, performing all the desirable low level decisions like path finding in an autonomous way. On his part, the Instructor is able to control all the crowd behaviour parameters in the simulation in real time.
Three main modules have been implemented in SAFECITI:
• A crowd simulation server: holds the core of the crowd simulation developed (all the Crowd and Police entities, simulates the effects of the actions and reactions which take place during an exercise), is able to give all the clients (Instructor and Trainees) the current state of the simulation and receive simulation commands to act on the simulation.
• Instructor: allows the trainer to load new basemaps, create scenarios for those basemaps, and prepare exercises for each one of those scenarios. Also allows to start, stop and monitor the simulation in real time, issuing high-level actions to the crowds present in the simulation, through 2D and 3D views of the real time simulation. Last, the Instructor is able to load previously ran simulations, which are recorded in a database, and debrief the actions taken by the Trainees.
• Trainee: allows trainees to control the Police units and equipment assigned by the Instructor and give their police units high-level commands which can affect the crowds, also through 2D and 3D views of the real time simulation.
Crowd Simulation
The first phase in the Crowd Simulation development was the design and implementation of the basic features of the Crowd Simulation Engine by Golaem. They started from their current developed technology, the Golaem SDK which included two basic components: Golaem Core and Golaem Path. Though this SDK some functionality which could be re-used in SAFECITI was already available, like multi-core entities simulation, Behaviours scheduling, simple behaviours (like GoTo), Pathfinding or Collision Avoidance. Some code rewriting was then performed to suit the specificities of the SAFECITI project in terms of scale. A lot of restructuring of the current Golaem SDK was also accomplished in order to decouple the simulation from the visualization. GO assembled these software components in a specific Crowd Simulation Engine for the SAFECITI project in order to expose the planned features to external users through a C-Style API. During the first year, the implementation of the Golaem Crowd Engine could be integrated to:
• Create a scenario through specific API calls and serialize the scenario in a binary buffer or file.
• Use the buffer to initialize the simulation.
• Update the simulation with a user given time step.
• Access simulation data in real time: number of groups and entities, positions of groups and entities, type of groups, etc.
• Basic behaviors to go to a user specified location, flee from a user given location, or deploy a formation (police groups only).
• Possibility to place barriers to block access to a navigable area.
• Logging to console and/or to file.
During second phase of the project the main focus was on developing the autonomous behaviours of the crowd with different configurable profiles, the propagation of aggressiveness through the crowd, the use of weapons by the crowd and the police. Golaem implemented a version of Influence Map to minimize memory use, optimize performance and suit the project’s needs. We use three different Influence Maps, to manage three different levels of information: Police Influence Map which stores the aggressiveness of the Police and is used by the crowds to select their behaviours, the Civilian Influence Map which stores the aggressiveness of civilians and is used to propagate aggressiveness through crowds with a “mood” algorithm and the Weapons Influence Map which stores the aggressiveness generated by weapons and is used by both factions to calculate stress and aggressiveness.
Different crowd profiles were developed to simulate different behaviours to the same given situation:
• Civilians with Standard profile will increase their AGG over time based on their suggestibility and surrounding AGG. Their own computed AGG can never exceed MAGG. If the perceived AGG is higher than their MAGG, they will leave the simulation and just “go home”.
• Civilians with Pacifist profile will never increase their AGG. If the perceived AGG is higher than their MAGG they will leave the simulation and just “go home”.
• Civilians with Instigator profile start by increasing their AGG normally like in Standard profile. But if AGG keeps growing, it will increase faster and faster, exponentially until it quickly equals MAGG. Instigators never flee the simulation.
• Civilians with Resistant profile act exactly like Standard profile, but never flee the simulation.
• Civilians with Docile profile increase their AGG normally like in the Standard profile but they will reduce their own AGG to its initial value if the perceived AGG is higher than the MAGG.
To create a realistic behaviour of protesting civilians, we needed to represent the overall tension within the crowd and make it spread between the protesters. We also needed to make the civilians perceive the tension around them generated by other civilians, police units and weapons. And finally civilians had to react to the tension according to their behavioural profile. To achieve this, Golaem built on the Influence Maps architecture to create the SAFECITI AGG Map, which is used as an Influence Map, but also as a Visibility Map and a Potential Field. This allowed them to solve the represent/perceive/react problems with one single memory and CPU efficient data structure.
As a result, a Crowd Simulation engine was developed meeting the performance requirements and the feature specifications of the project. It can simulate in real-time the behaviour of Crowds with different profiles and their reaction to Police actions controlled by the instructor. The engine has been delivered through a comprehensive and well documented C-Style API that has been successfully integrated into the SAFECITI Application.
Urban data generation
In the first stages of SAFECITI, partners were able to design and implement data harmonization (algorithms to map the main data source, analyzed with already existing software, onto the Esri 3D City Information Model scheme which contains all the required data for SAFECITI) and the procedural environment model generation, applying the procedural modelling system present in Esri CityEngine onto the Esri 3D City Information Model. Environment data like digital terrain model, building block outlines, parcel outlines, building footprint components and attributes (like the number of floors), points of interest and street walkable areas were identified and tested.
The dataflow and architecture of the Urban Generation Module was designed, also the SAFECITI Basemap format was specified, implementing the initial version and functionalities of the UGM API. The module was successfully integrated in the SAFECITI application and its functionalities tested, confirming that the data source (Spanish Government urban GIS “CATASTRO”) was usable to produce rich 3D environments for SAFECITI: 2d geometry of loaded cities for overview, 3D building models scenarios with several Levels of Detail and terrain information allowed plain 2D urban environments to become real time 3D environments. After the initial part of the project, several goals were achieved: SAFECITI was able to use quality-assured GIS data, the generated urban environments generated had a convincing outcome, with a look and level of detail which could change depending on the analysts needs, and enabling crowd simulations to access full 3D urban information model. This goals were achieved thanks to an extensive set of extensions specific for SAFECITI.
The UGM allows to read GIS databases on a specified schema and transform that data into SAFECITI GIS database schema (Basemap) and is able to generate 3D urban city models based in that transformed GIS databases, including the adjustment of the look and level of detail. Due to MIR-DGP suggestions, the SAFECITI workflow focused on a police training tool with little interaction with the environment, so the requirements to produce exact virtual replicas of existing cities and the need for real-time regeneration of the models were reduced. Instead, the focus shifted to abstract representations of existing locations, so the implemented functionalities gave place to models with minimum visual cues to establish a sense of locality and proportion.
The features UGM implemented for SAFECITI include, regarding data transformation into SAFECITI format: reading common GIS data formats, terrain generation, intermediate geometry generation for the for the navigation mesh generator implemented by the Crowd Simulation Engine, sidewalk geometry extraction based on street outlines. The procedural modelling engine for 3D urban environments allows also procedural modelling of transformed data, multiple levels of detail, and simplified geometry on demand and geometry encoding to ease the graphics module scenario rendering.
Several basemaps were also prepared for SAFECITI (Barcelona, Madrid, Valencia, Berlin and Toulouse), implementing access to three different basemap schemas (Spain’s CATASTRO, Germany’s Berlin 3D format and France’s OSM LGF).
Communications
SAFECITI application is an asymmetric online serious game, where each connected participant has a well defined function. All communications between participants have been implemented using DDS (Data Distribution Service), which is a machine to machine middleware standard that aims to enable scalable, real time, high-performance and interoperable data exchanges between publishers and subscribers.
During a simulation, there are, at least, three machines running: the crowd simulation server, the instructor and a trainee. The crowd simulation server has to send all the information about entities positions, velocities and status each frame to all the participants. The maximum number of entities that can be in a single simulation is 10,000 and the maximum number of trainees is 12. This means that information about 10,000 entities has to be sent to 13 participants (12 trainees and the instructor) each frame (60 times per second). To reduce the amount of bandwidth used to disseminate this information and to reduce positions latency between the server and the participants, a dead reckoning algorithm has been implemented by Next Limit. This algorithm calculates, on a very simple way, how each entity is going to keep moving based on previous information about position and velocity. If the result of this calculus is higher than a previously set threshold, the server sends an update with the real position and velocity of each entity that is not correctly placed. As most people keep their velocity during a while, the number of updates sent from the server is reduced to a 10% using the dead reckoning algorithm.
As the instructor is the one who manage the simulation, any action taken by him/her has to be sent to the server and the trainees. The instructor machine contains also the database where the exercise is being recording. Any action taken by trainees has to be received and recorded for debriefing and to train the predictive engine. Trainees and the instructor have each one a virtual radio that can be used to talk between them. These audio data is also sent using DDS.
Rendering
Starting from a generic 3D model of a male protester with two levels of detail, constraints were established for the future crowd rendering engine: number of bones, animations and polygons. Using the Next Limit development framework, a graphical system was directly implemented inside the SAFECITI project. Several algorithms for rendering large amounts of 3D avatars were tried, finally implementing a high performance character rendering engine, focusing on optimization and support for more than one viewport and camera of the same scene. The visual style of the application was designed, taking a clay render-like approach.
After an efficient crowd rendering system was implemented, more functionalities and optimizations were implemented for crowd rendering: different crowd avatars were added (male and female child, adult and old age models), all character models had three different levels of detail to improve rendering speed of distant models, police equipment was also added (each 3D model had also its own level of detail) and animations to reproduce the different behaviours implemented by the Crowd Rendering Engine were added, implementing a simple graph state to reproduce for each entity the behaviour being simulated by the Crowd Engine. To improve performance and save GPU memory, several features were added, like frustum culling, LOD rendering, vertex skinning to render those characters near the camera with the most detailed level of detail.
The scenario data also was highly optimized, performing a spatial partitioning of the static geometry generated by the Urban Generation Module, storing spatial partitioned data to directly be able to load and render each scenario without the need of any geometric post processing. Spatial partitioning also brought some features like ray casting to be able to interact (select) crowds and Points Of Interest by clicking on any entity / POI from any 2D and 3D camera position. Some features like zooming and camera transitions and crowd / police following were implemented to help the user get immersed in the simulation, also camera collision information was added to the scenario and some functionality to be able to focus POIs without having buildings between the camera and the POI was implemented. With that same goal, a couple of visual pleasing graphical effects were implemented (High Dynamic Range rendering and a view-independent shadow mapping technique known as Cascaded Shadow Mapping).
Although vehicle simulation amongst crowds is a complex task for the Crowd Rendering Engine and was not the main subject of the project, 3D vehicle models were implemented, allowing Police forces to get into and out from those vehicles and offer at least a schematic approach of the simulation when Police forces move through the scenario and crowds even if the simulation is not 100% accurate.
Following the features released by the Urban Generation Module, street labels and building numbers 3D information was added to the scenario for the 2D camera.
All this work gave place to a highly efficient crowd and open scenario rendering engine, specially focused and efficient in the rendering of thousands of entities at real time frame rates. The simulation limits regarding the number of entities were limited to 10000 by the Crowd Simulation Engine, being able this crowd rendering engine to easily render much higher amounts of characters at real time frame rates, something not available in the main commercial and open source game / rendering engines.
AI training and debriefing
An Artificial Intelligence has been implemented using machine learning techniques of classification and regression based on previously trained situations. Data used for training is obtained after the debriefing of each simulation.
SAFECITI debriefing mode allows the instructor to evaluate the actions taken by trainees during a simulation by giving a score to each action. Everything useful for debriefing happening during a simulation is stored in a MongoDB database. On debriefing, this information is loaded and used to show the scenario on every instant of the simulation, when and who took which action and the evolution in time of the intensity of police forces and the aggressiveness of the crowd. Using this information, the instructor evaluates each action with three possible values: 1- Good, 0 – Not necessary but not bad, -1 – the action must not have been taken.
The artificial intelligence is trained based on previous scores. Several machine learning techniques are used to classify actions based on several parameters, as crowd density, crowd velocity, aggressiveness, number of police forces, equipment or intensity used. All this information is used to find similar situations during a simulation and recommend the best scored action that is similar to the scenario that is being simulated.
Potential Impact:
The main expected impacts of SAFECITI, including the socio-economic impact and the wider societal implications of the project so far, are:
1. Improve the police forces training. SAFECITI is the first simulation platform for police crowd management training. SAFECITI provides the Police forces with a tool able to support their training programs. It allows a faster training of the police officers as decrease the dependency of training in real situations. The exercise repetition and debriefing can lead to better training results and ease the definition of training procedures and programs.
2. Speed up the process to introduce new crowd control techniques. Police and security forces can try and test new crowd control techniques and protocols virtually, implying low risk.
3. Improvement in the safety of citizens against different threats. The SAFECITI project will improve the safety conditions of people in public events, under situations that compromise the security by provoked violence or to deescalate situations on the brink of violence. The improvements in analyst’s skills will induce a potential benefit for citizens. The SAFECITI platform will set the basis of future add-ons to enrich the simulations with more complex systems to train the analysts on typical situations but also very rare high-impact events.
4. Threat anticipation and adequate response through planning. SAFECITI can help to anticipate the results of a wide range of threats, not only in public events with threats of violence, but also in man-made disasters.
5. Development of virtual intelligence analysts. SAFECITI is the First simulation platform for police crowd management training. The SAFECITI platform will not only serve to train analysts or improve their skills, but to create a modern analyst profile. Therefore the project will contribute to increase the technology level of the European police forces and will attract a number of specialists interested in both intelligence analysis and technology.
6. Technologies and services for new SME’s business development. This project will let the SME involved to start new business developments based on serious game technology for Defence and Security.
Furthermore, the Schengen EU is a single security space, with largely the same “black swan” threats relevant to each member state. At the same time, training, equipment and approaches are different from country to country. The SAFECITI system will allow for a better exchange of practices and approaches among European Police and security organizations, and enable more joint training programs.
Dissemination activities and exploitation of results
The dissemination efforts in the project have increased from the beginning. A Dissemination plan was defined during the first months of the project, and updated during the project life.
Due to the characteristics of the SAFECITI project there was a necessity for dealing with the confidentiality of data and all security related issues in relation to the project outcomes. For this purpose, a Security Committee was constituted. Confidentiality agreements DGP were signed to grant the security of very sensible information.
The Consortium Agreement included detailed provisions regarding Intellectual Properties and use rights. The allocation of activities easies the identification of the produced foreground and the IPRs holders.
For the SMEs involved in the consortium the project meant an excellent way to approach the security industry. Potential partners for commercializing the product of the project were contacted. Discussions have increased during the last stage of the project due to the lack of a functional prototype during the first months of the second period. These discussions will continue after the end of the project and could lead to agreements for the exploitation of the results of the project. On the other side, the Spanish national Police, as end user, has expressed their interest in further developments of the product.
The platform has been shown to potential stakeholders related to the security industry and security forces. Other channels of dissemination have been the project website, procedures for internal communication, papers and scientific dissemination, presentations in conferences and events aimed at the Security industries, and direct contacts with stakeholders.
The project logo and website were designed at the beginning of the project (www.SAFECITI.eu). The website includes general information about the project and the partners involved, a section to contact the consortium and news regarding activities of the Consortium.
As scientific dissemination, several papers were issued and relevant events were attended giving lectures:
- BIOMAT 2014, June 2014, Granada, Spain.
- Ki-Net, October 2014, Philadelphia, USA.
- Academy of Science Morocco, November 2014, Rabat, Morocco.
- ICMMA2014, January 2015, Tokyo, Japan.
- CAMS, February 2015, Paris, France.
- Conference in KAU University, March 2015, Jeddah, Saudi Arabia.
- “Advances in Computational Fluid- Structure Interactions and Flow Simulations” Conference. May 2015, Istanbul, Turkey.
- Lecture at University of Ferrara, Italy, November 2015.
- Laboratoire Jacques-Louis Lions (UPMC). Paris, France, December 2015.
- Organization for Economic Co-operation and Development (OECD). Paris, France, January 2016.
- Lecture at MOX (Laboratory for Modeling and Scientific Computing). February 2016, Milan, Italy.
- PhD Course “Mathematical Tools for Complex Systems: Models, Simulations, and Analytic Problems”. Turin. Italy, February 2016.
- Laboratoire Jacques-Louis Lions (UPMC). Paris, France, December 2015.
Papers:
- N. Bellomo, F. Colasuonno, D. Knopoff, J. Soler , “From a systems theory of sociology to modeling the onset and evolution of criminality”, Networks and Heterogeneous Media 10 (3), 421-441, 2015.
- G. Ajmone Marsan, N. Bellomo, L. Gibelli, “Stochastic Evolving Differential Games Toward a Systems Theory of Behavioral Social Dynamics”, accepted for publication in Mathematical Models and Methods in Applied Sciences, 2015.
- D. Burini, S. De Lillo, L. Gibelli, “Collective learning modeling based on the kinetic theory of active particles”, accepted for publication in Physical of Life Reviews, 2015.
- N. Bellomo, L. Gibelli, “Behavioral Crowds: Modeling and Monte Carlo Simulations Toward Validation”, submitted to Computers & Fluids.
SAFECITI has been presented in several congress related to the security field:
- Smart City Expo World Congress (Barcelona, Spain, November 2014)
- I/ITSEC Expo (Orlando, USA, December 2014)
- HOMSEC (Madrid, Spain, March 2015)
- MILIPOL Paris (Paris, France, November 2015)
- International course "Technological Innovation in policing” (Avila, Spain, November 2015)
- CEPOL Conference “New perspectives of cooperation between practice, education and police science” (Lisbon, Portugal, October 2015)
The attendance to Milipol Paris resulted in the following publications:
• Presentation of SAFECITI in a Milipol video report on an online newspaper dedicated to the French Military Police (Gendarmerie Nationale) audience. http://www.lessor.org/video-lantiterrorisme-au-coeur-du-salon-milipol/
• Article on SAFECITI on 01net.com http://www.01net.com/actualites/drones-applis-simulateurs-les-meilleures-innovations-high-tech-pour-l-armee-et-la-police-932316.html
• Cover picture and half page inside the MILIPOL News issue 1 distributed to all MILIPOL participants.
List of Websites:
www.safeciti.eu
Main Contact
Next Limit
Calle Angel Cavero 2
Madrid (Spain)
email: safeciti@nextlimit.com
A fully functional platform has been developed according to the requisites exposed by the Spanish Police forces. The platform is designed for training purpose and integrates different modules. The main results are:
- Crowd simulation module. The system is able to simulate the behavior of crowds of up to 10.000 people.
- Urban Generation module. The system include the maps of real cities. The maps are generated with the information available of the street structure of the cities, so it is a realistic representation of the cities.
- User profiles. The system includes different interfaces and different functionalities for each profile of users (instructor or trainee)
- Adaptability. The system can be adapted to different security forces as it allows to edit important parameters according to the end users’ procedures.
- Communications. The platform includes the functionality of communication by chat and by voice, recreating realistic conditions of the operations.
- Predictive capability. The system recommends actions during an exercise based on previous similar situations
Dissemination efforts during this period were aimed at promoting the project in international events, in particular Milipol Paris. The platform has been shown to potential stakeholders related to the security industry. Other channels of dissemination have been the project website, procedures for internal communication, papers and scientific dissemination, presentations in conferences and events aimed at the security industries, and direct contacts with security forces and stakeholders. The project got the feedback from external participants through an Ethical Committee and an Advisory Board. Procedures to securely exchange confidential information have been agreed and respected. A Security Committee has controlled the dissemination.
Effective project management allowed the most cost-effective use of the EU contribution and produced the complete achievement of the results for the project.
Project Context and Objectives:
SAFECITI is a simulation platform that addresses the requirements of the civil security intelligence analysis community for the specific problem of crowd safety and anticipated threat control in cities. The platform is not only a “game” in the context of serious game technology, but an advanced simulator that aims to predict the complex behaviour of the crowd and to provide the analysts with new technology to improve the tactics and operations that are used in real situations.
This platform is designed both as a simulation platform for training purposes and as a predictive tool for operational use. The platform also includes measurement tools to evaluate the analysts based on their results. All the simulations will be stored in a database that will work as an historic repository, but more interestingly, as a large set of useful data that will be re-analysed by artificial intelligence algorithms to create predictive models. This predictive model will create a virtual analyst that is able to recommend actions based on the enriched analysis of hundreds or thousands of simulations.
The development of the Simulation Platform involves several technological challenges:
- Interactive graphical platform and interfaces and control system: development of the main control system, integration of all the modules in the interactive platform; user interfaces; scene setup; interactive workflow and control and network modules.
- Graphics Components: 3D models for human representation, vehicles and city infrastructure
- Generation of urban environment: development of an urban generation module to generate real city layouts based on real GIS data sources, e.g. from Madrid.
- Advanced crowd simulation: design and development of the crowd simulation engine that realistically simulates crowds of virtual humans, their behaviours and their actions on the environment.
- Database module and predictive model: development of database components and predictive models.
The main objectives of the project are:
• Intelligence analysis enhancement. Strengthening the performance and efficiency of intelligence analysts by means of new technologies.
• Simulation of people behaviour. The realistic simulation of crowd behaviours is one of the central aspects of the SAFECITI project.
• Interactive serious game platform. The crowd simulation engine will be integrated in the interactive serious game platform, and will be controlled using a specifically designed user interface.
• Predictive engine. The development of a predictive model will help the intelligence analysts in the decision-making processes.
Project Results:
The SAFECITI (Safe Citizen) Project was a two year project from 2014 to 2016, joining the efforts of six partners from academic to purely practical fields from different countries in the EU. The goal of the SAFECITI project was to develop a real-time application able of simulating realistic crowd behaviour in response to police actions during a crowd control scenario. Presented as an asymmetric online serious game, were an Instructor is able to control all the crowds behaviours in a scenario and several Trainees with an assigned set of Police officers and equipment have to deal with the situations generated by the Instructor and take the correct tactical decisions.
Entities simulating the police units are able to be commanded at high level, performing all the desirable low level decisions like path finding in an autonomous way. On his part, the Instructor is able to control all the crowd behaviour parameters in the simulation in real time.
Three main modules have been implemented in SAFECITI:
• A crowd simulation server: holds the core of the crowd simulation developed (all the Crowd and Police entities, simulates the effects of the actions and reactions which take place during an exercise), is able to give all the clients (Instructor and Trainees) the current state of the simulation and receive simulation commands to act on the simulation.
• Instructor: allows the trainer to load new basemaps, create scenarios for those basemaps, and prepare exercises for each one of those scenarios. Also allows to start, stop and monitor the simulation in real time, issuing high-level actions to the crowds present in the simulation, through 2D and 3D views of the real time simulation. Last, the Instructor is able to load previously ran simulations, which are recorded in a database, and debrief the actions taken by the Trainees.
• Trainee: allows trainees to control the Police units and equipment assigned by the Instructor and give their police units high-level commands which can affect the crowds, also through 2D and 3D views of the real time simulation.
Crowd Simulation
The first phase in the Crowd Simulation development was the design and implementation of the basic features of the Crowd Simulation Engine by Golaem. They started from their current developed technology, the Golaem SDK which included two basic components: Golaem Core and Golaem Path. Though this SDK some functionality which could be re-used in SAFECITI was already available, like multi-core entities simulation, Behaviours scheduling, simple behaviours (like GoTo), Pathfinding or Collision Avoidance. Some code rewriting was then performed to suit the specificities of the SAFECITI project in terms of scale. A lot of restructuring of the current Golaem SDK was also accomplished in order to decouple the simulation from the visualization. GO assembled these software components in a specific Crowd Simulation Engine for the SAFECITI project in order to expose the planned features to external users through a C-Style API. During the first year, the implementation of the Golaem Crowd Engine could be integrated to:
• Create a scenario through specific API calls and serialize the scenario in a binary buffer or file.
• Use the buffer to initialize the simulation.
• Update the simulation with a user given time step.
• Access simulation data in real time: number of groups and entities, positions of groups and entities, type of groups, etc.
• Basic behaviors to go to a user specified location, flee from a user given location, or deploy a formation (police groups only).
• Possibility to place barriers to block access to a navigable area.
• Logging to console and/or to file.
During second phase of the project the main focus was on developing the autonomous behaviours of the crowd with different configurable profiles, the propagation of aggressiveness through the crowd, the use of weapons by the crowd and the police. Golaem implemented a version of Influence Map to minimize memory use, optimize performance and suit the project’s needs. We use three different Influence Maps, to manage three different levels of information: Police Influence Map which stores the aggressiveness of the Police and is used by the crowds to select their behaviours, the Civilian Influence Map which stores the aggressiveness of civilians and is used to propagate aggressiveness through crowds with a “mood” algorithm and the Weapons Influence Map which stores the aggressiveness generated by weapons and is used by both factions to calculate stress and aggressiveness.
Different crowd profiles were developed to simulate different behaviours to the same given situation:
• Civilians with Standard profile will increase their AGG over time based on their suggestibility and surrounding AGG. Their own computed AGG can never exceed MAGG. If the perceived AGG is higher than their MAGG, they will leave the simulation and just “go home”.
• Civilians with Pacifist profile will never increase their AGG. If the perceived AGG is higher than their MAGG they will leave the simulation and just “go home”.
• Civilians with Instigator profile start by increasing their AGG normally like in Standard profile. But if AGG keeps growing, it will increase faster and faster, exponentially until it quickly equals MAGG. Instigators never flee the simulation.
• Civilians with Resistant profile act exactly like Standard profile, but never flee the simulation.
• Civilians with Docile profile increase their AGG normally like in the Standard profile but they will reduce their own AGG to its initial value if the perceived AGG is higher than the MAGG.
To create a realistic behaviour of protesting civilians, we needed to represent the overall tension within the crowd and make it spread between the protesters. We also needed to make the civilians perceive the tension around them generated by other civilians, police units and weapons. And finally civilians had to react to the tension according to their behavioural profile. To achieve this, Golaem built on the Influence Maps architecture to create the SAFECITI AGG Map, which is used as an Influence Map, but also as a Visibility Map and a Potential Field. This allowed them to solve the represent/perceive/react problems with one single memory and CPU efficient data structure.
As a result, a Crowd Simulation engine was developed meeting the performance requirements and the feature specifications of the project. It can simulate in real-time the behaviour of Crowds with different profiles and their reaction to Police actions controlled by the instructor. The engine has been delivered through a comprehensive and well documented C-Style API that has been successfully integrated into the SAFECITI Application.
Urban data generation
In the first stages of SAFECITI, partners were able to design and implement data harmonization (algorithms to map the main data source, analyzed with already existing software, onto the Esri 3D City Information Model scheme which contains all the required data for SAFECITI) and the procedural environment model generation, applying the procedural modelling system present in Esri CityEngine onto the Esri 3D City Information Model. Environment data like digital terrain model, building block outlines, parcel outlines, building footprint components and attributes (like the number of floors), points of interest and street walkable areas were identified and tested.
The dataflow and architecture of the Urban Generation Module was designed, also the SAFECITI Basemap format was specified, implementing the initial version and functionalities of the UGM API. The module was successfully integrated in the SAFECITI application and its functionalities tested, confirming that the data source (Spanish Government urban GIS “CATASTRO”) was usable to produce rich 3D environments for SAFECITI: 2d geometry of loaded cities for overview, 3D building models scenarios with several Levels of Detail and terrain information allowed plain 2D urban environments to become real time 3D environments. After the initial part of the project, several goals were achieved: SAFECITI was able to use quality-assured GIS data, the generated urban environments generated had a convincing outcome, with a look and level of detail which could change depending on the analysts needs, and enabling crowd simulations to access full 3D urban information model. This goals were achieved thanks to an extensive set of extensions specific for SAFECITI.
The UGM allows to read GIS databases on a specified schema and transform that data into SAFECITI GIS database schema (Basemap) and is able to generate 3D urban city models based in that transformed GIS databases, including the adjustment of the look and level of detail. Due to MIR-DGP suggestions, the SAFECITI workflow focused on a police training tool with little interaction with the environment, so the requirements to produce exact virtual replicas of existing cities and the need for real-time regeneration of the models were reduced. Instead, the focus shifted to abstract representations of existing locations, so the implemented functionalities gave place to models with minimum visual cues to establish a sense of locality and proportion.
The features UGM implemented for SAFECITI include, regarding data transformation into SAFECITI format: reading common GIS data formats, terrain generation, intermediate geometry generation for the for the navigation mesh generator implemented by the Crowd Simulation Engine, sidewalk geometry extraction based on street outlines. The procedural modelling engine for 3D urban environments allows also procedural modelling of transformed data, multiple levels of detail, and simplified geometry on demand and geometry encoding to ease the graphics module scenario rendering.
Several basemaps were also prepared for SAFECITI (Barcelona, Madrid, Valencia, Berlin and Toulouse), implementing access to three different basemap schemas (Spain’s CATASTRO, Germany’s Berlin 3D format and France’s OSM LGF).
Communications
SAFECITI application is an asymmetric online serious game, where each connected participant has a well defined function. All communications between participants have been implemented using DDS (Data Distribution Service), which is a machine to machine middleware standard that aims to enable scalable, real time, high-performance and interoperable data exchanges between publishers and subscribers.
During a simulation, there are, at least, three machines running: the crowd simulation server, the instructor and a trainee. The crowd simulation server has to send all the information about entities positions, velocities and status each frame to all the participants. The maximum number of entities that can be in a single simulation is 10,000 and the maximum number of trainees is 12. This means that information about 10,000 entities has to be sent to 13 participants (12 trainees and the instructor) each frame (60 times per second). To reduce the amount of bandwidth used to disseminate this information and to reduce positions latency between the server and the participants, a dead reckoning algorithm has been implemented by Next Limit. This algorithm calculates, on a very simple way, how each entity is going to keep moving based on previous information about position and velocity. If the result of this calculus is higher than a previously set threshold, the server sends an update with the real position and velocity of each entity that is not correctly placed. As most people keep their velocity during a while, the number of updates sent from the server is reduced to a 10% using the dead reckoning algorithm.
As the instructor is the one who manage the simulation, any action taken by him/her has to be sent to the server and the trainees. The instructor machine contains also the database where the exercise is being recording. Any action taken by trainees has to be received and recorded for debriefing and to train the predictive engine. Trainees and the instructor have each one a virtual radio that can be used to talk between them. These audio data is also sent using DDS.
Rendering
Starting from a generic 3D model of a male protester with two levels of detail, constraints were established for the future crowd rendering engine: number of bones, animations and polygons. Using the Next Limit development framework, a graphical system was directly implemented inside the SAFECITI project. Several algorithms for rendering large amounts of 3D avatars were tried, finally implementing a high performance character rendering engine, focusing on optimization and support for more than one viewport and camera of the same scene. The visual style of the application was designed, taking a clay render-like approach.
After an efficient crowd rendering system was implemented, more functionalities and optimizations were implemented for crowd rendering: different crowd avatars were added (male and female child, adult and old age models), all character models had three different levels of detail to improve rendering speed of distant models, police equipment was also added (each 3D model had also its own level of detail) and animations to reproduce the different behaviours implemented by the Crowd Rendering Engine were added, implementing a simple graph state to reproduce for each entity the behaviour being simulated by the Crowd Engine. To improve performance and save GPU memory, several features were added, like frustum culling, LOD rendering, vertex skinning to render those characters near the camera with the most detailed level of detail.
The scenario data also was highly optimized, performing a spatial partitioning of the static geometry generated by the Urban Generation Module, storing spatial partitioned data to directly be able to load and render each scenario without the need of any geometric post processing. Spatial partitioning also brought some features like ray casting to be able to interact (select) crowds and Points Of Interest by clicking on any entity / POI from any 2D and 3D camera position. Some features like zooming and camera transitions and crowd / police following were implemented to help the user get immersed in the simulation, also camera collision information was added to the scenario and some functionality to be able to focus POIs without having buildings between the camera and the POI was implemented. With that same goal, a couple of visual pleasing graphical effects were implemented (High Dynamic Range rendering and a view-independent shadow mapping technique known as Cascaded Shadow Mapping).
Although vehicle simulation amongst crowds is a complex task for the Crowd Rendering Engine and was not the main subject of the project, 3D vehicle models were implemented, allowing Police forces to get into and out from those vehicles and offer at least a schematic approach of the simulation when Police forces move through the scenario and crowds even if the simulation is not 100% accurate.
Following the features released by the Urban Generation Module, street labels and building numbers 3D information was added to the scenario for the 2D camera.
All this work gave place to a highly efficient crowd and open scenario rendering engine, specially focused and efficient in the rendering of thousands of entities at real time frame rates. The simulation limits regarding the number of entities were limited to 10000 by the Crowd Simulation Engine, being able this crowd rendering engine to easily render much higher amounts of characters at real time frame rates, something not available in the main commercial and open source game / rendering engines.
AI training and debriefing
An Artificial Intelligence has been implemented using machine learning techniques of classification and regression based on previously trained situations. Data used for training is obtained after the debriefing of each simulation.
SAFECITI debriefing mode allows the instructor to evaluate the actions taken by trainees during a simulation by giving a score to each action. Everything useful for debriefing happening during a simulation is stored in a MongoDB database. On debriefing, this information is loaded and used to show the scenario on every instant of the simulation, when and who took which action and the evolution in time of the intensity of police forces and the aggressiveness of the crowd. Using this information, the instructor evaluates each action with three possible values: 1- Good, 0 – Not necessary but not bad, -1 – the action must not have been taken.
The artificial intelligence is trained based on previous scores. Several machine learning techniques are used to classify actions based on several parameters, as crowd density, crowd velocity, aggressiveness, number of police forces, equipment or intensity used. All this information is used to find similar situations during a simulation and recommend the best scored action that is similar to the scenario that is being simulated.
Potential Impact:
The main expected impacts of SAFECITI, including the socio-economic impact and the wider societal implications of the project so far, are:
1. Improve the police forces training. SAFECITI is the first simulation platform for police crowd management training. SAFECITI provides the Police forces with a tool able to support their training programs. It allows a faster training of the police officers as decrease the dependency of training in real situations. The exercise repetition and debriefing can lead to better training results and ease the definition of training procedures and programs.
2. Speed up the process to introduce new crowd control techniques. Police and security forces can try and test new crowd control techniques and protocols virtually, implying low risk.
3. Improvement in the safety of citizens against different threats. The SAFECITI project will improve the safety conditions of people in public events, under situations that compromise the security by provoked violence or to deescalate situations on the brink of violence. The improvements in analyst’s skills will induce a potential benefit for citizens. The SAFECITI platform will set the basis of future add-ons to enrich the simulations with more complex systems to train the analysts on typical situations but also very rare high-impact events.
4. Threat anticipation and adequate response through planning. SAFECITI can help to anticipate the results of a wide range of threats, not only in public events with threats of violence, but also in man-made disasters.
5. Development of virtual intelligence analysts. SAFECITI is the First simulation platform for police crowd management training. The SAFECITI platform will not only serve to train analysts or improve their skills, but to create a modern analyst profile. Therefore the project will contribute to increase the technology level of the European police forces and will attract a number of specialists interested in both intelligence analysis and technology.
6. Technologies and services for new SME’s business development. This project will let the SME involved to start new business developments based on serious game technology for Defence and Security.
Furthermore, the Schengen EU is a single security space, with largely the same “black swan” threats relevant to each member state. At the same time, training, equipment and approaches are different from country to country. The SAFECITI system will allow for a better exchange of practices and approaches among European Police and security organizations, and enable more joint training programs.
Dissemination activities and exploitation of results
The dissemination efforts in the project have increased from the beginning. A Dissemination plan was defined during the first months of the project, and updated during the project life.
Due to the characteristics of the SAFECITI project there was a necessity for dealing with the confidentiality of data and all security related issues in relation to the project outcomes. For this purpose, a Security Committee was constituted. Confidentiality agreements DGP were signed to grant the security of very sensible information.
The Consortium Agreement included detailed provisions regarding Intellectual Properties and use rights. The allocation of activities easies the identification of the produced foreground and the IPRs holders.
For the SMEs involved in the consortium the project meant an excellent way to approach the security industry. Potential partners for commercializing the product of the project were contacted. Discussions have increased during the last stage of the project due to the lack of a functional prototype during the first months of the second period. These discussions will continue after the end of the project and could lead to agreements for the exploitation of the results of the project. On the other side, the Spanish national Police, as end user, has expressed their interest in further developments of the product.
The platform has been shown to potential stakeholders related to the security industry and security forces. Other channels of dissemination have been the project website, procedures for internal communication, papers and scientific dissemination, presentations in conferences and events aimed at the Security industries, and direct contacts with stakeholders.
The project logo and website were designed at the beginning of the project (www.SAFECITI.eu). The website includes general information about the project and the partners involved, a section to contact the consortium and news regarding activities of the Consortium.
As scientific dissemination, several papers were issued and relevant events were attended giving lectures:
- BIOMAT 2014, June 2014, Granada, Spain.
- Ki-Net, October 2014, Philadelphia, USA.
- Academy of Science Morocco, November 2014, Rabat, Morocco.
- ICMMA2014, January 2015, Tokyo, Japan.
- CAMS, February 2015, Paris, France.
- Conference in KAU University, March 2015, Jeddah, Saudi Arabia.
- “Advances in Computational Fluid- Structure Interactions and Flow Simulations” Conference. May 2015, Istanbul, Turkey.
- Lecture at University of Ferrara, Italy, November 2015.
- Laboratoire Jacques-Louis Lions (UPMC). Paris, France, December 2015.
- Organization for Economic Co-operation and Development (OECD). Paris, France, January 2016.
- Lecture at MOX (Laboratory for Modeling and Scientific Computing). February 2016, Milan, Italy.
- PhD Course “Mathematical Tools for Complex Systems: Models, Simulations, and Analytic Problems”. Turin. Italy, February 2016.
- Laboratoire Jacques-Louis Lions (UPMC). Paris, France, December 2015.
Papers:
- N. Bellomo, F. Colasuonno, D. Knopoff, J. Soler , “From a systems theory of sociology to modeling the onset and evolution of criminality”, Networks and Heterogeneous Media 10 (3), 421-441, 2015.
- G. Ajmone Marsan, N. Bellomo, L. Gibelli, “Stochastic Evolving Differential Games Toward a Systems Theory of Behavioral Social Dynamics”, accepted for publication in Mathematical Models and Methods in Applied Sciences, 2015.
- D. Burini, S. De Lillo, L. Gibelli, “Collective learning modeling based on the kinetic theory of active particles”, accepted for publication in Physical of Life Reviews, 2015.
- N. Bellomo, L. Gibelli, “Behavioral Crowds: Modeling and Monte Carlo Simulations Toward Validation”, submitted to Computers & Fluids.
SAFECITI has been presented in several congress related to the security field:
- Smart City Expo World Congress (Barcelona, Spain, November 2014)
- I/ITSEC Expo (Orlando, USA, December 2014)
- HOMSEC (Madrid, Spain, March 2015)
- MILIPOL Paris (Paris, France, November 2015)
- International course "Technological Innovation in policing” (Avila, Spain, November 2015)
- CEPOL Conference “New perspectives of cooperation between practice, education and police science” (Lisbon, Portugal, October 2015)
The attendance to Milipol Paris resulted in the following publications:
• Presentation of SAFECITI in a Milipol video report on an online newspaper dedicated to the French Military Police (Gendarmerie Nationale) audience. http://www.lessor.org/video-lantiterrorisme-au-coeur-du-salon-milipol/
• Article on SAFECITI on 01net.com http://www.01net.com/actualites/drones-applis-simulateurs-les-meilleures-innovations-high-tech-pour-l-armee-et-la-police-932316.html
• Cover picture and half page inside the MILIPOL News issue 1 distributed to all MILIPOL participants.
List of Websites:
www.safeciti.eu
Main Contact
Next Limit
Calle Angel Cavero 2
Madrid (Spain)
email: safeciti@nextlimit.com
