Final Report Summary - FORLAB (Forensic Laboratory for in-situ evidence analysis in a post blast scenario)
Executive Summary:
The post blast scene in an IED-based attack covers a wide area were very small debris of the explosion are spread. Those debris can be easily missed by the specialists and differentiating general debris from “clue evidences” is a hard process that currently cannot be performed in field. Instead, a huge number of samples are usually collected in field and analyzed in one of the few existing specialized laboratories.
The main objective of the ForLab (www.fp7-ForLab.eu) is to increase effectiveness of the forensic teams providing a new tool to localize hidden evidences, discriminate real evidences from genral debris, improving the capability to recreate the scenario and making the information about the investigation readily available to a commander in almost real time.
The project activities have been broken down into eleven work packages and are distributed across 36 months from March 2012 to February 2015.
The ForLab system is composed by the following main elements:
• Communication and positioning subsystem: To provide robust encrypted communication and accurate automatic position of the collected evidences
• Real time 3D scene recreation subsystem: To provide a navigable 3D model of the scene where the evidences are represented
• Command and control centre: To make all the information available on the investigation readily available in real time to a commander.
• Searching and screening technologies
o LIF (Laser Induced Fluorescence) scanning system: To help to locate plastic and polymeric debris
o LIBS and Raman field analysis system to discriminate samples containing traces of the explosive from non relevant samples
o NLJD system: To help to localize electronic debris in the scene.
During the second week of September 2013, field tests of the individual technologies were carried out at a military training ground in Wroclaw (Poland). A short movie of this event has been prepared and uploaded to the ForLab website ("News and Events").
Currently all the modules of the final system are available and have been integrated into a "ForLab system" that has been tested during one week in the Centre of Operational Practices of CNP located in Linares (south of Spain) during the last week of January 2015.
ForLab was successfully tested in three simulated, almost real scenarios: (1) explosion in a metro/train station, (2) mail delivered IED into an office, and (3) vehicle borne IED. The test week ended with workshop with End user of several European countries including a live demonstration of the use of ForLab
Project Context and Objectives:
Usually the post blast scenario of an IED attack consists of a wide area covered by very small debris of the explosion. Every detail may be of importance for the identification of the terrorist group responsible for the attack, the person who emplaced the explosive or even the person that assembled the IED. However, differentiating general debris from “clue evidences” is a hard process that, today cannot be performed in the field and therefore a huge number of samples are collected in field and sent to a distant laboratory to perform a deep analysis to determine its relevance. As the relevance of the evidences is unknown to the investigators, they trend to collect larger amount of evidences (to ensure the success) thus generating a huge amount of work for the laboratories.
Any knowledge about the author of the attack or the procedures they used can be of vital importance to prevent new attacks. Comparison of the characteristics of the attack with previous incidents may also help and the fast exploitation of this information may dramatically improve the results.
The quality of analysis depends on the quality of recognition, documentation, collection and preservation of the evidence. So, the current needs of the end users include sample detection, identification, capture of the scene and a kit suitable for semi-automatic crime scene reconstruction supported by a positioning system.
ForLab is a novel systematic methodology which fulfils the end user demands by:
• On site detection, identification and analysis of evidences, providing the investigator with objective criteria for the selection of the samples.
• Producing a real time 3D recreation of the scene including the localization of the evidences
• Making all relevant information of the scene readily available in a Command and Control Centre to the expert leading the investigation, and making this information also available to the investigators.
The final goal is to optimize the evidence collection and to reduce the time and resources in the laboratory, while preserving the chain of custody so as to minimize the time required to identify the responsible for the attack.
The on-site detection, identification, analysis of evidences in real time will let the investigator to pre-evaluate the evidences and to decide which of them must be collected. The optical spectroscopies are good candidates to screening post-blast IED evidences due to their capability to analyze different types of evidences in few seconds and to operate in field. Often, the fragments of the triggering electronics of the IED are the most valuable evidences for the identification of the authors, NLJD has the necessary characteristics for the easy localisation of those fragments.
The following technologies have been developed in ForLab:
LIBS (Laser Induced Breakdown Spectroscopy): LIBS is a very sensitive technique that provides information on the elemental composition of the analytes, and has demonstrated its capability to identify tenths of nanograms of explosives over a surface in just a few seconds.
RAMAN spectroscopy: This technology measures directly the characteristic vibrational spectrum of the sample under investigation. This molecular specific fingerprint allows direct identification of the substance under investigation.
Laser-induced fluorescence (LIF) is one of the most sensitive detection schemes. The fluorescence is the light spontaneously emitted due to transitions from excited singlet states to various vibrational levels of the electronic ground state. The experimental measure of the fluorescence emitted by matter carries information about both the photophysical properties of the molecule and the chemical and physical nature of its micro surroundings. In general terms the LIF is not compound specific; however the additional capability of active reflectance measurements represents a significant step forward in the identification and precise localization of debris dispersed all around the crime scene.
NLJD: This technology is based on the intrinsic property of non linear junctions such as those contained in semi-conductors, to radiate different harmonics, when radiated at a given frequency. It provides the ability to detect electronics, even if switched off or in sleeping mode.
In addition, this tools will integrated into the ForLab system providing additional tools for management of the information acquired from the scene.
3D modelling technology has been developed to allow the fast generation of a 3D model of the scene providing a real time dynamic overview of the scene.
Non GPS based localization technology capable of work in indoor and outdoor scenarios has been developed to provide the accurate localization (10 cm) of the evidences within the 3D model of the scene.
A secure wireless communication network ensures the reliable transmission of the information to the Command a Control centre where the data received from the different sensors is received and presented to the operator in a comprehensible way.
Al the evidences collected are registered using a Tablet PC integrated into the ForLab network that allows the generation of an evidence report that is transmitted to the command centre, ensuring the preservation of the chain of custody.
FORLAB components will be deployed into the scene of the explosion of an IED as follows:
The first elements to be deployed will be the communication network and the 3D scene recreation tool that will make a 3D model of the scene available at the Command and Control Centre in just about 30 minutes. Initially a low resolution model is acquired and transmitted to the command and control centre that will decide if it is necessary to increase the level of detail of the complete scene or a priority area.
The second element to be deployed will be the LIF analysis tool that will start a scan of the scene from a fixed point scanning an area (up to 30 m. from the sensor), looking for potential evidences. LIF images of the scene highlighting the presence of plastics, and polymeric debris transmitted to the command and control centre and combined with the 3D view of the scene in just about 15 minutes.
Finally, the screening technologies (LIBS, RAMAN and NLJD) will be deployed:
NLJD will help the technicians performing a visual inspection of the scene to localize hidden debris of electronic devices. The localization of those debris will be clearly identified in the 3D model at the Command and Control Centre.
LIBS and Raman will be used to get an chemical analysis of the samples identified by the investigators. The automated recognition software of the sensor will provide the investigation with an immediate objective indication of the presence of explosive traces on the selected sample.
Every time the investigator decides to collect a sample, a report of the evidence is created in a Tablet PC (including pictures, localization, time of collection, etc), digitally signed (to ensure the integrity and the authoring) and sent to the command and control centre.
Figure 1 Operational diagram of the FORLAB
The command and control centre may be physically located near the explosion scene or may be located at a distant place. This gives the ForLab system the major advantage of allowing highly qualified investigators to follow the investigation from the command and control centre and to direct investigators in the scene to do sampling in specific areas of higher relevance or to instruct them to perform some specific tests on particular samples.
ForLab will provide end users with:
• 3D scene recreation for improved understanding of the scene during investigation and for later recreation of the scene for further investigations
• Four quick and portable screening tools to detect and identify forensic evidences in field
• Automated sample localization in the scenario
• A command and control centre where the evidence information can be processed in almost real time and depicted in the 3D scenario
• Possibility to recreate the events registered by the Command and control centre during the investigation for training proposes or to review the steps of the investigation.
FORLAB project is addressing the topic SEC 2010.1.3-2 “Forensic Analysis of an explosion or an unexploded IED”. It relates to the problem of evidence collection in the post-blast scene, providing the investigators with a new tool compatible with the exiting procedures for this kind of investigations.
The main focus of the project has been.
• Provide the investigators an objective criteria of evaluation of samples to reduce the number of evidences to be collected and sent to the reference laboratory.
• Improve the capability of recreating the scenario to help on the real time identification of areas of higher interest and helping on the recreation of the scene for later investigations.
• Make the information on the investigation available to the investigators in real time and in a comprehensive way.
The involvement of end user was also a major objective since the beginning of the project. This objective was supported by the presence of the security forces of four European countries as members of the consortium that have largely influenced the final results of the project.
Project Results:
FORLAB introduces several aspects of novelty:
• Although the selected analysis technologies are already existing technologies, all of them needed to be further developed to be fielded and to match the requirements of the forensic investigators at the post-blast scene.
• 3D imaging is also an existing technology already in use by security forces to document and investigate crime scenes but the objective of the ForLab was to develop this technology to make possible the generation and transmission of the 3D model of the scene in almost real time.
• ForLab introduces a new concept for the analysis of the post-blast scene based on real time availability of all the information gathered from the scene while the chain of custody of evidences is preserved.
The following paragraphs summarize achievements of the project on each of the technologies.
LIF as an screening technology
The fluorescence is the light spontaneously emitted due to transitions from excited singlet states to various vibrational levels of the electronic ground state.
Thanks to new technological development, Fluorescence Spectroscopy (FS) is now a widely used scientific tool promoted from the scientific field to a routine method for real time analysis in several experimental fields as biochemistry, biophysics, material sciences and forensic studies.
A typical experimental fluorescence sensor device, based on laser-induced fluorescence (LIF), is shown in Figure 1, with its main component being an optical radar, able to detect the emission induced by UV laser on remote surfaces; thus, several portions of the surface exposed to the laser light can be analyzed with high spatial resolution.
Figure 1 Components of laser induced fluorescence sensors for remote measurement
With specific reference to the analysis of a post blast scenario, the LIF technique can be used to identify special kind of debris having specific spectral signature, like parts of printed circuit boards and plastic material. Literature studies have shown that in general terms the LIF is not compound specific, however the additional capability of active reflectance measurements represents a significant step forward in the identification and precise localization of debris dispersed all around the crime scene. To fulfil these additional goals which are not specific of LIF system, the sensor will operate in at least two different modes by slightly changing the device set up (reflectance and fluorescence measurements).
Indeed the reflectance and fluorescence operating modes share most of the optoelectronic equipment, namely the collection optics and the spectral detector, while differing on one hand in the way the sample is excited, while on the other hand they differ in the way the spectral detector is operated.
Active reflectance measurements
Active reflectance measurements are based on elastic backscattering of the incident laser light: a spatial scan is performed while the laser is on. The collection optics is focused on to the scanned scene, and the acquisition camera operates with a high attenuation filter to avoid the saturation of the optical sensor. The result gives for each pixel of the scanned area the monochromatic reflectance intensity. Reflecting surfaces at normal incidence and irregular metallic debris as well are identified by proper processing algorithms.
Fluorescence measurements
The measurement of fluorescence spectra induced by laser is made by performing a scan while the laser light is on. The system uses a patent protected procedure to acquire spectra also in full day light: in this case a proprietary method is used to discriminate between the light induced from the laser (LIF signal) with respect to the light diffused by the area under study and by the environment. The result gives the fluorescence spectrum for each pixel of the scanned area; then a successive data analysis of the acquired spectrum provides detailed information on the area under study.
Analysis of spectral measurements
To speed up the data analysis on the acquired images, the most relevant spectral features are identified by Principal Component Analysis (PCA). Although it is commonly admitted that the PCs do not possess any direct physical meaning, they can nevertheless be represented as spectra suitable to be described in terms of bands. In some cases a given PC has a well defined spectroscopic band with associated peaks, while in other cases more complicated trends and shapes are observed: most frequent is the case of a band set against another. Few of the PC components are usually retained for subsequent analysis: typically 5 to 8 components are enough to describe the entire spectral data set. It is also worth noticing the possibility to build suitable linear combinations of the computed PCs to have a faithful representation of each pixel spectra, eventually used for the computation of standard CIE/lab colorimetric measurement.
The PCA devoted to the identification of prominent spectral features, relieves from the lengthy time consuming examination of each of the spectra acquired and can considerably shorten the time need to communicate with the Control Centre. The advantages of this procedure are that it is fast and can run in a semiautomatic mode, having though the inconvenience of requiring a global analysis, possibly ignoring those local peculiarities which do not possess enough statistical significance. To overcome this drawback local PCA can also be performed on different portions of the scanned areas, and then the results can be analyzed separately. Once identified, spectral bands are sought for in the acquired LIF spectra, completing the data analysis.
A different method used in the analysis of spectral images, concerns the identification of regions having a specific spectral content. Typical is the case of identification of a given pigment in an image: such tasks are accomplished either by a band analysis, or by using spectral mapper algorithms like SAM (Spectral Angle Mapper) or SCM (Spectral Correlation Mapper). Although the mapper algorithms perform well with a low computational cost, their performances are generally lower with respect to the band analysis procedures.
The LIF scanning system developed in ForLab has the following characteristics:
• Image scan (64m2 at 10 m)
• Field of view 38 x 38 deg
o angular resolution 0.02deg
o spatial resolution 0.5 cm at 10m
o CCD area 20x20 mm
• laser excitation prr up to 500Hz
• 100μJ/cm2 at 10m
• Multispectral sensor
• scan time 8min/image
• weight 40Kg
• size 37 x 34 x 70 cm
• Fast data processing (< 5min)
Figure 2. LIF scanning system
LIBS and Raman as screening technologies
LIBS is an atomic emission spectroscopy technique used for the real-time, nondestructive determination of elemental composition and requires no sample preparation. The technique relies on the microplasma created by a focused laser pulse, typically several nanoseconds in length, to dissociate molecules and particulates within the plasma volume. The subsequent emission can be resolved spectrally and temporally to generate a spectrum containing emission lines from the atomic, ionic, and molecular fragments created by the plasma. The LIBS technique yields detailed information on elemental compositions, including many minor and trace elements.
Nowadays, laser-induced breakdown spectroscopy (LIBS) has been shown to be a suitable technology for detecting the presence of organic material and in some cases determining the type of organic material. Applying LIBS to energetic organic material detection and identification is of interest for various applications, including force protection, security concerns, forensic analysis, etc. Other LIBS applications are analysis of environmental, archaeological, geological materials and use in artwork analysis.
Raman spectroscopy is a spectroscopic technique used to study vibrational, rotational, and other low-frequency modes in a system. Typically, a sample is illuminated with a laser beam. Light from the illuminated spot is collected with a lens and sent through a monochromator. Wavelengths close to the laser line, due to elastic Rayleigh scattering, are filtered out, while the rest of the collected light is dispersed onto a detector. Chemical species that exhibit a change in polarizability with vibration exhibit Raman spectra that are uniquely determined by their vibrational mode structure. Analytical techniques based on Raman spectroscopy have been widely used for explosive detection and characterization. Raman has been demonstrated for analyzing minerals, use in artwork analysis, etc.
In the ForLab project a new portable system combining LIBS and Raman analysis capability has been be designed and developed with the following characteristics:
Figure 3. LIBS-Raman system developed in ForLab
• Two modes of operation as table-top equipment or as backpack to be able to access and analyze samples that are difficult to reach
• Capability to detect different types of explosives directly on different surfaces or using a swab
• Analysis time, including automated response in less than one minute
• There is no contamination of the detector and therefore continuous operation is possible
• Designed to avoid cross contamination between samples.
• Capable of automated detection of particles of 90 ng of explosive with LIBS and particles with diameter of 300 µm with Raman
• Easy to update Raman substance database for identification
• Capability of detection of gunshot residues (GSR) on standard sampling kits and other surfaces, providing a reliable result in less than one minute.
During the final tests of the project the LIBS system achieved outstanding results being able to differentiate samples containing residues of the explosives from samples not containing residues (or in a quantity below the limit of detection of the system) in all the three scenarios.
B 1.2.4 NLJD for detection of electronic debris
The foundation technology (NLJD) was previously developed to detect small electronic components used in eavesdropping systems and electronic bugs and has been well known for over 30 years. This technology is based on the intrinsic property of non linear junctions such as those contained in semi-conductors, to radiate different harmonics, when radiated at a given frequency. It provides the ability to detect electronics, even if switched off or in sleeping mode.
The electronic components are detected in real time by measurement of the 2nd and the 3rd harmonic levels. Localization of electronic debris will be obtained using a man-portable NLJD detector. When the operator detects an electronic device, he pushes a button to inform the Command and Control Center that the current NLJD detector position within the 3D scene corresponds to the position of an electronic device. The position of the NLJD detector will be determined thanks a 3D scene reconstruction performed by a dedicated platform implemented with a LIDAR (Astrium’s contribution to the Forlab system). This platform is man-portable and has to be moved to different positions so that the LIDAR measurements can be combined to build up the 3D scene.
The precise position of the NLJD detector into the 3D scene, taking into account the sweeping movement of the operator scrutinizing the area will be obtained by a Kinect camera.
Figure 4 The full system with the NLJD scanner
B 1.2.5 3D Scene modelling for forensics
The start of any forensic investigation must begin with the collection, review, and analysis of evidence. The better the quality of evidence, the better the analysis and likelihood of solving the crime is.
Although photogrammetric techniques using cameras, tripods are regularly used, the trend today is to use directly 3D laser Scanning technologies. It is an ever growing and useful application of laser based measurement technologies in fighting crimes and reconstructing events.
Often referred to as High Definition Surveying (HDS), 3D scanning became popular in the late 1990s. Some scanning technologies are based on optical methods whereby photographs are used to collect and match points in corresponding photographs (i.e. Photogrammetry and stereo matching) while the lesser known CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) allow the interior structures of objects to be "scanned" and examined.
The more common 3D Lidar (Light Detection and Ranging) scanners emit a beam of light and measure the part of the beam that is reflected back to the instrument. These are the most common types of scanners being able to collect and preserve data from very large crime scenes.
The two most common types of Lidar scanners are “phase-based” and “pulse-based” scanners, which refer to the method for determining the distance to any surface that has been scanned.
Compared to classical approaches HSL techniques, allow acquiring data in detail without predetermining what is and what is not evidence. It allows scanning organic shapes and highly curved surfaces that would otherwise be difficult to measure (ex of bloodstains evidence spanning over several surfaces, furniture, and walls).It can physically reach point of measurement without touching, deteriorating or contaminating it. Most measurements can be done from a safe distance but equipment operators and the general public should be at safe eyes distances. real-time visual feedback as the points are being captured, are possible making the scene immediately available for review and analyzed at the scene. Areas lacking detail can then be re-scanned while other areas that did not scan properly can be addressed.
However, level of Required Accuracy depends on the practical range in given conditions. Some reflective surfaces, dark/highly absorptive surfaces or oblique surfaces may not provide any or good data. Registration errors, when combining several scans in one large point cloud, need to be considered.
There is now a migration towards photorealistic 3D environments where millions of points can be measured and analyzed. 3D scanning is still relatively new and costly so that other capable technologies such as photogrammetry, should be also considered as well as combination of the two.
The greatest area of development will be in the available software tools for the analysis and visualization of scan data. Some of these tools apply to bullet trajectories, bloodstain pattern analysis, and even "image projection" where a suspect's height can be estimated from security videos. Once the analysis is complete, the results can be immediately displayed as a 3D animation or through an interactive viewer for court presentations. Eventually, the benefits of 3D scanners and related analysis tools will become more familiar to law enforcement agencies, attorneys, and jurors.
The ForLab project developed the capability to produce 3D models directly on the field in an iterative way.
Figure 5. 3D modelling tool developed in ForLab
The Innovations of the approach are:
• Acquiring rapidly 3D models and not simply 3D surfaces or 3D geometric points, with the challenge of processing the data directly on the spot in order to deliver near real time data to the Command & Control Centre.
• Acquiring not only geometry and radiometry, but also semantics and eventually topology
• Allow easy integration (geo-localisation) of any complementary detections and measures without needing specific set-up on the scene (position reference patterns...).
• Allow rapid deployment through acquisition planning on the spot and rapid set up: typical deployment timing would be around 10 minutes, with acquisition lasting a few minutes, and the redeployment taking a few extra minutes.
• Specific tool to optimise the acquisition process to limit the number of acquisitions while preserving minimum occlusions.
• Ability to produce even from data with occlusions, complete 3D model with occluded areas interpreted: even if we do not see the floor behind the chair, or the wall behind the car, we can rebuild the floor plane or the wall plane to produce a complete 3D model.
B1.2.6 Sample position reference indoor and outdoor
Autonomous tracking and localization approaches that do not rely on GPS are becoming very attractive the last years especially for indoor navigation. The main characteristics of autonomous tracking systems, are deployment time, accuracy, portability and reliability. Moreover these systems ought to be pervasive and require zero interaction with the users.
Depending on the target application different requirements for each characteristic are needed, and as a result, radically different systems can be developed that depend on different working principles and are of different cost. The core design choice that serves different requirements, is the point –to-point distance measurement. This can take place by inferring distance, using physics laws, from measurable natural phenomena like sound time-of-flight, electromagnetic signal time-of-flight, electromagnetic signal angle of arrival, electromagnetic signal strength. Depending on the physics formulas that associate the aforementioned measures with distance, the achievable level of accuracy is determined. For instance, time-of-flight can be easily associated with distance because speed of sound and speed of light are well measured, stable and a simple formula associates them with distance. On the other hand, signal strength depends on a variety of factors that is very difficult or impossible to measure, and the distance association approach is to take indicative measurements and implement curve-fitting maths. The latter approach radically affects the deployment time because it requires a training stage, where the user needs an additional different method to measure the distance in order to train the system and accuracy can vary at runtime because several factor can affect received signal strength. But, there is no free lunch, what is straightforward to compute using robust physics formulas is very hard to measure in practice. More specifically, the light speed of electromagnetic signal requires clocks with picoseconds accuracy so that reasonable accuracy (10cm) is achievable. Current state of the art processors cannot compete with required performance, so application specific hardware accelerators are required to implement baseband processing which along with very sophisticated algorithms result in very expensive devices. In ForLab UTH implemented 2 approaches for the measurement of distance between 2 points:
Option 1: Received Signal strength of 802.11n (CSI metric) for low-end devices
Option 2: Signal Time of Flight using Ultra wideband(UWB) which was used in the final system and demonstrated in Linares.
These diverse systems achieved the following performance for distance measurement:
Main specifications of option 1 RSS:
• deployment needs training (20 min)
• 60 meters is the maximum distance that can be measured between 2 points.
• Accuracy of 1-2 meters
• Very cheap (30 € per device)
Main specifications of option 2 ToF:
• deployment does not need training
• 120 meters is the maximum distance
• Accuracy of 10 cm
• Relatively expensive (600€ per device)
Another important subsystem of the localization module is the tracking solver which uses distances and known anchor Cartesian coordinates to provide the Cartesian coordinates of a point on any mapping system. The main job of this module is to solve a system of non-linear equations that associate the given point with the known anchors. The math are standard and based on Non-linear least square solving approaches which come in several flavours. UTH thoroughly explored several localization areas solution space using different solving algorithms to determine their performance and appropriateness for the specific type of problem. Moreover, they have developed a novel methodology to detect a faulty distance measurement when the number of anchors is more than 3 and exclude the faulty measurement for the calculations. The approach is based on solving all possible combinations in pairs of 3 and inspect the convergence effort which indicates when a problematic measurement affects the solution region. The specific algorithm detects faulty measurements in 90% of the cases and is in the core of a patent application.
The final ForLab system field requirements lead to the adoption of option 2 for distance measurement. More specifically, the final system specifications are as follows:
• Position Accuracy 10cm
• Automated alignment with a any coordinates reference
• Minimum deployment requires 4 anchors (arbitrary placement)
• Minimum coverage 10.000 square meters (100m x 100m area)
• Response time depends on a number of concurrent devices.
• Each device introduces a 20msec delay for (1 second for 5 devices)
• 8 hours non-stop operation on batteries
• Weight of a ruggedized full set with batteries, 4 anchors and 4 location devices is 5kg.
• Prototype cost of a full set 12.000€ (8 devices).
• Average deployment time 15 minutes (needs trained personnel)
• Proven field operation (under rain)
Figure 6. Anchor points and positioning module in the office scenario
B1.2.7 Command and control centre (Knowledge Management)
“Command and Control” addresses the arrangement of personnel, equipment, communications, facilities, and procedures employed by a commander or a command center in planning, directing, coordinating, and controlling resources and operations in the accomplishment of a mission.
“Operations” deals with the way the operational process and procedures are conducted by users and deciders using many data sources to support decisions and to optimize results.
“Communications” looks into the way the data and commands are shared and transmitted among many sources and users linked together to support decisions.
“Knowledge Management” consists of:
• Sensor fusion for prediction and estimate of system internal states by optimal stochastic filtering of system and sensor dynamic models with real measurements, and
• Data integration, management, merging, analysis and exploitation of the multiple sources of data to support decisions and how deciders can interact with the data.
Command and Control Center
A Command and Control Center is typically a secure facility that operates as dispatch, surveillance monitoring, alarm and coordination office center. A command and control center that is used in a deployed location is usually called a command post.
Command and control functions are performed through an arrangement of personnel, equipment, communications, facilities, and procedures employed by a commander in planning, directing, coordinating, and controlling operations in the accomplishment of a mission.
Commanding officers are assisted by specialized staff officers and personnel. These staff provides a bi-directional flow of accurate, timely information between a commanding officer and subordinate field teams, which by category represents information on which command decisions are based.
The key application is that of decisions that effectively manage resources. While information flow toward the commander is a priority, information that is useful or contingent in nature is communicated forward to lower staffs and field teams.
The ForLab project developed plug-ins to:
• Enable forensic investigation professionals shall be guided through the scene.
• Enable presenting an overview of the scene that the professionals are involved in.
• Enable presenting possible evolutions of the situation based on the collected data.
Using visual navigation for after action analysis shall help:
• Professionals to perform more detailed debriefing of data and procedures leading to improved future operations.
• Better prevention and mitigation of risks.
The main novelties of the ForLab project with respect to command and control centre will be to:
• Bring state of the art decision making command and control techniques using:
o Dynamic modeling of data collection sensors supporting command and control centre
o Explore and model detailed application requirements that shall drive technology development
• Bring into the system data and information management for collected data reduction, browsing and rapid access.
• Bring system indexing of large scale distributed data
Data Integration and analysis
A data system must also be capable of integration, management, merging, analysis and exploitation of data, which in turn can be very complex due to the multiple nature of the different data and their sources. Besides, conflicts and multiple-access are also very common in the operational scenario due to wrong data, multiple sources and source conflicts in redundant data.
Data fusion is composed of a set of subtasks that filter, transform, reorganize and optimize data. Common Data fusion operators are: Minimum union, Full Disjunction, Complement Union, Merge and Priority Merge, Match Join and Group and aggregation
Although the engineering literature is replete with examples of how data fusion techniques are being applied in military and industry projects, they are just now beginning to be applied to many kinds of projects. Their integration is a challenge in order not only to optimize the data produced by them, but to improve their own work process.
The main novelties of the ForLab project with respect to data fusion, integration and analysis will be to address the:
• Server
o AMQP message broker (RabbitMQ)
o Key-based distributed data store (RiakCS)
o Application modules coordinated through AMQP
o REST service
• Client
o Eclipse RCP based product
o 3D renderer and scene management
o Graphing, multi-windowing, cross-platform
• Capabilities:
o 3D scene with in-situ evidences and aerial view
o Data visualizations for each sensor
o Rich client and mobile client
• Use of data:
o Replay investigations
o Simulated investigations
• Integration:
o Standards based messaging and plug-in architectures
o Distributed processing and analysis
Figure 7. Screenshot of the C&CC. Metro station scenario
Potential Impact:
As defined in the work programme, the expected impact of the topic SEC-2011.1.3-2 is the following
“To help law enforcement agencies in their analysis and to deal with explosive events in a common European approach”
ForLab has delivered a novel systematic methodology for optimizing the process of evidence collection and analysis. We aspire that our approach shall maximize the speed, reliability and accuracy of the process and ultimately make a significant step forward in the battle against terrorism reducing the time needed to identify the authors of the attack and increasing the effectiveness of the response of the Security Forces.
Today a new concept on the fight against the terrorism is widely accepted. This concept is based in the idea that the explosion of and IED is the final step in a chain of events driving any IED based attack:
1. The attack must be planned and financed
2. The people to perform the attack must be recruited
3. The IED must be prepared, a supply chain is necessary to support the construction of the IED
4. The IED must be emplaced
5. Finally the IED must be detonated at the desired time.
As we approach the step five, the chances to prevent the attack are reduced. Every time that there is an explosion of an IED it is of vital importance to retrieve as much information as possible about the inspirers of the attack, the materials used for the preparation of the device, the specific characteristics of the attack (emplacement, amount of explosive, etc) any clue that may reveal information about the previous steps that may lead to disrupt the chain of events of the next terrorist attack.
The time of reaction is a critical aspect of the fight against terror, the sooner the security forces can gain the knowledge about the author, the greater the chances of success. The information retrieved from the scene must be ready to be shared with Security Forces of friendly countries so that the terrorist and their support chain can be pursued beyond the border of the country suffering the attack.
The objective of the ForLab project is to reduce the time of response of the Security Forces by improving the existing capabilities for the analysis of the post-blast scene of an IED attack. This objective can be achieved based on the following aspects:
• Providing the investigations with tools to help on the localization of potential evidences
• Providing the investigators with tools for the objective evaluation of the quality of the evidences collected.
• Making information available immediately and in comprehensive way in the Command and control centre, making possible information sharing with other security body.
• Allowing expert investigators to guide less experienced investigators on the field form the remote Command and control centre.
• Proposing a new concept to approach the investigation of the scene based in full availability of the information gathered from the scene in real time to drive the investigation, and combine this information with the available information from previous events of similar characteristics. The success of this approach will be multiplied the proposed solution matches the procedures in use by Security Forces of different countries of Europe.
Focussed development of existing technologies from different countries in Europe to help Security Forces in the selection and localization of the evidences and the coalition of Security Forces from several countries with industry is only possible in the context of a European programme. ForLab is a good example o this cooperation giving as results some tools highly tailored to the common need of European security forces.
In particular, the achievements of the project in detection of traces of explosives will have very positive impact on the work of the laboratories of analysis in the investigation of a post-blast event:
- on the economic aspect : to be able to discriminate on the field between the elements and to select some of them more relevant, allows more effectiveness but also avoid to make a great number of expensive (and useless) analysis in laboratories thus reducing the costs and saving valuable time.
- Although all the results get on the field must to be confirm by a police (or a specialised) laboratory: the "analytical-field-discrimination" allows to concentrate the first effort on some relevant samples which gave positive answer (about traces of explosives) on the field.
If a first and relevant selection of the elements is made on the field, it allows to avoid the "jam effect" for the forensics laboratories. Indeed, if too much samples arrived at the same time to the labs with no priority given, we have a risk to "paralyse" the system, particularly in case of multiple attacks
Although ForLab was very focussed on the post-blast scene, during the execution of the project we have seen that most of the technologies and concept developed in ForLab can be applied to other crime scenes. Making a exhaustive description of the possible application of ForLab it is out of the scope and extension of this document, but we can just outline a few of those applications.
• The concept of making the electronic report on the collected evidences is exportable to virtually any crime scene.
• Te concept of creating the 3D model and 2D map of the scene and place the evidences on the map is already used in different crime scenes, but ForLab allows doing it in real time
• Getting the position of samples in an automated way (even not having the 3D model) will save a valuable of time in investigations with huge amount of evidences
• The LIBS and Raman systems have the capability to detect (or identify) virtually any substance, and the concept of a portable and reliable equipment can be exported to other crime scenes involving, to mention just a few, drugs, gunshot residues, chemical agents...
• In a similar way the LIF technology is applicable not only to the detection of polymeric debris, and the concept of analysing a an area from a standoff distance can be exported to other scenarios
• The command and control centre has been designed as an open architecture, making easy the integration of new tools that the security forces may be using in the future.
In particular, we must highlight that as a result of the project, two companies (Airbus Defence and Space and Indra) have ongoing projects to bring to the market a 3D modelling tool for forensic applications and a portable LIBS system for forensic applications with expectative to get them available within the next year.
Contact data:
Indra Sistemas S.A
F. Javier Hernández
Tlf: +34 987849888
e-mail: fjhernandez@indra.es
University of Thessaly, Greece
Dr. Dimitris Syrivelis
Tlf: +30 6944318012
e-mail: jsyr@inf.uth.gr
The post blast scene in an IED-based attack covers a wide area were very small debris of the explosion are spread. Those debris can be easily missed by the specialists and differentiating general debris from “clue evidences” is a hard process that currently cannot be performed in field. Instead, a huge number of samples are usually collected in field and analyzed in one of the few existing specialized laboratories.
The main objective of the ForLab (www.fp7-ForLab.eu) is to increase effectiveness of the forensic teams providing a new tool to localize hidden evidences, discriminate real evidences from genral debris, improving the capability to recreate the scenario and making the information about the investigation readily available to a commander in almost real time.
The project activities have been broken down into eleven work packages and are distributed across 36 months from March 2012 to February 2015.
The ForLab system is composed by the following main elements:
• Communication and positioning subsystem: To provide robust encrypted communication and accurate automatic position of the collected evidences
• Real time 3D scene recreation subsystem: To provide a navigable 3D model of the scene where the evidences are represented
• Command and control centre: To make all the information available on the investigation readily available in real time to a commander.
• Searching and screening technologies
o LIF (Laser Induced Fluorescence) scanning system: To help to locate plastic and polymeric debris
o LIBS and Raman field analysis system to discriminate samples containing traces of the explosive from non relevant samples
o NLJD system: To help to localize electronic debris in the scene.
During the second week of September 2013, field tests of the individual technologies were carried out at a military training ground in Wroclaw (Poland). A short movie of this event has been prepared and uploaded to the ForLab website ("News and Events").
Currently all the modules of the final system are available and have been integrated into a "ForLab system" that has been tested during one week in the Centre of Operational Practices of CNP located in Linares (south of Spain) during the last week of January 2015.
ForLab was successfully tested in three simulated, almost real scenarios: (1) explosion in a metro/train station, (2) mail delivered IED into an office, and (3) vehicle borne IED. The test week ended with workshop with End user of several European countries including a live demonstration of the use of ForLab
Project Context and Objectives:
Usually the post blast scenario of an IED attack consists of a wide area covered by very small debris of the explosion. Every detail may be of importance for the identification of the terrorist group responsible for the attack, the person who emplaced the explosive or even the person that assembled the IED. However, differentiating general debris from “clue evidences” is a hard process that, today cannot be performed in the field and therefore a huge number of samples are collected in field and sent to a distant laboratory to perform a deep analysis to determine its relevance. As the relevance of the evidences is unknown to the investigators, they trend to collect larger amount of evidences (to ensure the success) thus generating a huge amount of work for the laboratories.
Any knowledge about the author of the attack or the procedures they used can be of vital importance to prevent new attacks. Comparison of the characteristics of the attack with previous incidents may also help and the fast exploitation of this information may dramatically improve the results.
The quality of analysis depends on the quality of recognition, documentation, collection and preservation of the evidence. So, the current needs of the end users include sample detection, identification, capture of the scene and a kit suitable for semi-automatic crime scene reconstruction supported by a positioning system.
ForLab is a novel systematic methodology which fulfils the end user demands by:
• On site detection, identification and analysis of evidences, providing the investigator with objective criteria for the selection of the samples.
• Producing a real time 3D recreation of the scene including the localization of the evidences
• Making all relevant information of the scene readily available in a Command and Control Centre to the expert leading the investigation, and making this information also available to the investigators.
The final goal is to optimize the evidence collection and to reduce the time and resources in the laboratory, while preserving the chain of custody so as to minimize the time required to identify the responsible for the attack.
The on-site detection, identification, analysis of evidences in real time will let the investigator to pre-evaluate the evidences and to decide which of them must be collected. The optical spectroscopies are good candidates to screening post-blast IED evidences due to their capability to analyze different types of evidences in few seconds and to operate in field. Often, the fragments of the triggering electronics of the IED are the most valuable evidences for the identification of the authors, NLJD has the necessary characteristics for the easy localisation of those fragments.
The following technologies have been developed in ForLab:
LIBS (Laser Induced Breakdown Spectroscopy): LIBS is a very sensitive technique that provides information on the elemental composition of the analytes, and has demonstrated its capability to identify tenths of nanograms of explosives over a surface in just a few seconds.
RAMAN spectroscopy: This technology measures directly the characteristic vibrational spectrum of the sample under investigation. This molecular specific fingerprint allows direct identification of the substance under investigation.
Laser-induced fluorescence (LIF) is one of the most sensitive detection schemes. The fluorescence is the light spontaneously emitted due to transitions from excited singlet states to various vibrational levels of the electronic ground state. The experimental measure of the fluorescence emitted by matter carries information about both the photophysical properties of the molecule and the chemical and physical nature of its micro surroundings. In general terms the LIF is not compound specific; however the additional capability of active reflectance measurements represents a significant step forward in the identification and precise localization of debris dispersed all around the crime scene.
NLJD: This technology is based on the intrinsic property of non linear junctions such as those contained in semi-conductors, to radiate different harmonics, when radiated at a given frequency. It provides the ability to detect electronics, even if switched off or in sleeping mode.
In addition, this tools will integrated into the ForLab system providing additional tools for management of the information acquired from the scene.
3D modelling technology has been developed to allow the fast generation of a 3D model of the scene providing a real time dynamic overview of the scene.
Non GPS based localization technology capable of work in indoor and outdoor scenarios has been developed to provide the accurate localization (10 cm) of the evidences within the 3D model of the scene.
A secure wireless communication network ensures the reliable transmission of the information to the Command a Control centre where the data received from the different sensors is received and presented to the operator in a comprehensible way.
Al the evidences collected are registered using a Tablet PC integrated into the ForLab network that allows the generation of an evidence report that is transmitted to the command centre, ensuring the preservation of the chain of custody.
FORLAB components will be deployed into the scene of the explosion of an IED as follows:
The first elements to be deployed will be the communication network and the 3D scene recreation tool that will make a 3D model of the scene available at the Command and Control Centre in just about 30 minutes. Initially a low resolution model is acquired and transmitted to the command and control centre that will decide if it is necessary to increase the level of detail of the complete scene or a priority area.
The second element to be deployed will be the LIF analysis tool that will start a scan of the scene from a fixed point scanning an area (up to 30 m. from the sensor), looking for potential evidences. LIF images of the scene highlighting the presence of plastics, and polymeric debris transmitted to the command and control centre and combined with the 3D view of the scene in just about 15 minutes.
Finally, the screening technologies (LIBS, RAMAN and NLJD) will be deployed:
NLJD will help the technicians performing a visual inspection of the scene to localize hidden debris of electronic devices. The localization of those debris will be clearly identified in the 3D model at the Command and Control Centre.
LIBS and Raman will be used to get an chemical analysis of the samples identified by the investigators. The automated recognition software of the sensor will provide the investigation with an immediate objective indication of the presence of explosive traces on the selected sample.
Every time the investigator decides to collect a sample, a report of the evidence is created in a Tablet PC (including pictures, localization, time of collection, etc), digitally signed (to ensure the integrity and the authoring) and sent to the command and control centre.
Figure 1 Operational diagram of the FORLAB
The command and control centre may be physically located near the explosion scene or may be located at a distant place. This gives the ForLab system the major advantage of allowing highly qualified investigators to follow the investigation from the command and control centre and to direct investigators in the scene to do sampling in specific areas of higher relevance or to instruct them to perform some specific tests on particular samples.
ForLab will provide end users with:
• 3D scene recreation for improved understanding of the scene during investigation and for later recreation of the scene for further investigations
• Four quick and portable screening tools to detect and identify forensic evidences in field
• Automated sample localization in the scenario
• A command and control centre where the evidence information can be processed in almost real time and depicted in the 3D scenario
• Possibility to recreate the events registered by the Command and control centre during the investigation for training proposes or to review the steps of the investigation.
FORLAB project is addressing the topic SEC 2010.1.3-2 “Forensic Analysis of an explosion or an unexploded IED”. It relates to the problem of evidence collection in the post-blast scene, providing the investigators with a new tool compatible with the exiting procedures for this kind of investigations.
The main focus of the project has been.
• Provide the investigators an objective criteria of evaluation of samples to reduce the number of evidences to be collected and sent to the reference laboratory.
• Improve the capability of recreating the scenario to help on the real time identification of areas of higher interest and helping on the recreation of the scene for later investigations.
• Make the information on the investigation available to the investigators in real time and in a comprehensive way.
The involvement of end user was also a major objective since the beginning of the project. This objective was supported by the presence of the security forces of four European countries as members of the consortium that have largely influenced the final results of the project.
Project Results:
FORLAB introduces several aspects of novelty:
• Although the selected analysis technologies are already existing technologies, all of them needed to be further developed to be fielded and to match the requirements of the forensic investigators at the post-blast scene.
• 3D imaging is also an existing technology already in use by security forces to document and investigate crime scenes but the objective of the ForLab was to develop this technology to make possible the generation and transmission of the 3D model of the scene in almost real time.
• ForLab introduces a new concept for the analysis of the post-blast scene based on real time availability of all the information gathered from the scene while the chain of custody of evidences is preserved.
The following paragraphs summarize achievements of the project on each of the technologies.
LIF as an screening technology
The fluorescence is the light spontaneously emitted due to transitions from excited singlet states to various vibrational levels of the electronic ground state.
Thanks to new technological development, Fluorescence Spectroscopy (FS) is now a widely used scientific tool promoted from the scientific field to a routine method for real time analysis in several experimental fields as biochemistry, biophysics, material sciences and forensic studies.
A typical experimental fluorescence sensor device, based on laser-induced fluorescence (LIF), is shown in Figure 1, with its main component being an optical radar, able to detect the emission induced by UV laser on remote surfaces; thus, several portions of the surface exposed to the laser light can be analyzed with high spatial resolution.
Figure 1 Components of laser induced fluorescence sensors for remote measurement
With specific reference to the analysis of a post blast scenario, the LIF technique can be used to identify special kind of debris having specific spectral signature, like parts of printed circuit boards and plastic material. Literature studies have shown that in general terms the LIF is not compound specific, however the additional capability of active reflectance measurements represents a significant step forward in the identification and precise localization of debris dispersed all around the crime scene. To fulfil these additional goals which are not specific of LIF system, the sensor will operate in at least two different modes by slightly changing the device set up (reflectance and fluorescence measurements).
Indeed the reflectance and fluorescence operating modes share most of the optoelectronic equipment, namely the collection optics and the spectral detector, while differing on one hand in the way the sample is excited, while on the other hand they differ in the way the spectral detector is operated.
Active reflectance measurements
Active reflectance measurements are based on elastic backscattering of the incident laser light: a spatial scan is performed while the laser is on. The collection optics is focused on to the scanned scene, and the acquisition camera operates with a high attenuation filter to avoid the saturation of the optical sensor. The result gives for each pixel of the scanned area the monochromatic reflectance intensity. Reflecting surfaces at normal incidence and irregular metallic debris as well are identified by proper processing algorithms.
Fluorescence measurements
The measurement of fluorescence spectra induced by laser is made by performing a scan while the laser light is on. The system uses a patent protected procedure to acquire spectra also in full day light: in this case a proprietary method is used to discriminate between the light induced from the laser (LIF signal) with respect to the light diffused by the area under study and by the environment. The result gives the fluorescence spectrum for each pixel of the scanned area; then a successive data analysis of the acquired spectrum provides detailed information on the area under study.
Analysis of spectral measurements
To speed up the data analysis on the acquired images, the most relevant spectral features are identified by Principal Component Analysis (PCA). Although it is commonly admitted that the PCs do not possess any direct physical meaning, they can nevertheless be represented as spectra suitable to be described in terms of bands. In some cases a given PC has a well defined spectroscopic band with associated peaks, while in other cases more complicated trends and shapes are observed: most frequent is the case of a band set against another. Few of the PC components are usually retained for subsequent analysis: typically 5 to 8 components are enough to describe the entire spectral data set. It is also worth noticing the possibility to build suitable linear combinations of the computed PCs to have a faithful representation of each pixel spectra, eventually used for the computation of standard CIE/lab colorimetric measurement.
The PCA devoted to the identification of prominent spectral features, relieves from the lengthy time consuming examination of each of the spectra acquired and can considerably shorten the time need to communicate with the Control Centre. The advantages of this procedure are that it is fast and can run in a semiautomatic mode, having though the inconvenience of requiring a global analysis, possibly ignoring those local peculiarities which do not possess enough statistical significance. To overcome this drawback local PCA can also be performed on different portions of the scanned areas, and then the results can be analyzed separately. Once identified, spectral bands are sought for in the acquired LIF spectra, completing the data analysis.
A different method used in the analysis of spectral images, concerns the identification of regions having a specific spectral content. Typical is the case of identification of a given pigment in an image: such tasks are accomplished either by a band analysis, or by using spectral mapper algorithms like SAM (Spectral Angle Mapper) or SCM (Spectral Correlation Mapper). Although the mapper algorithms perform well with a low computational cost, their performances are generally lower with respect to the band analysis procedures.
The LIF scanning system developed in ForLab has the following characteristics:
• Image scan (64m2 at 10 m)
• Field of view 38 x 38 deg
o angular resolution 0.02deg
o spatial resolution 0.5 cm at 10m
o CCD area 20x20 mm
• laser excitation prr up to 500Hz
• 100μJ/cm2 at 10m
• Multispectral sensor
• scan time 8min/image
• weight 40Kg
• size 37 x 34 x 70 cm
• Fast data processing (< 5min)
Figure 2. LIF scanning system
LIBS and Raman as screening technologies
LIBS is an atomic emission spectroscopy technique used for the real-time, nondestructive determination of elemental composition and requires no sample preparation. The technique relies on the microplasma created by a focused laser pulse, typically several nanoseconds in length, to dissociate molecules and particulates within the plasma volume. The subsequent emission can be resolved spectrally and temporally to generate a spectrum containing emission lines from the atomic, ionic, and molecular fragments created by the plasma. The LIBS technique yields detailed information on elemental compositions, including many minor and trace elements.
Nowadays, laser-induced breakdown spectroscopy (LIBS) has been shown to be a suitable technology for detecting the presence of organic material and in some cases determining the type of organic material. Applying LIBS to energetic organic material detection and identification is of interest for various applications, including force protection, security concerns, forensic analysis, etc. Other LIBS applications are analysis of environmental, archaeological, geological materials and use in artwork analysis.
Raman spectroscopy is a spectroscopic technique used to study vibrational, rotational, and other low-frequency modes in a system. Typically, a sample is illuminated with a laser beam. Light from the illuminated spot is collected with a lens and sent through a monochromator. Wavelengths close to the laser line, due to elastic Rayleigh scattering, are filtered out, while the rest of the collected light is dispersed onto a detector. Chemical species that exhibit a change in polarizability with vibration exhibit Raman spectra that are uniquely determined by their vibrational mode structure. Analytical techniques based on Raman spectroscopy have been widely used for explosive detection and characterization. Raman has been demonstrated for analyzing minerals, use in artwork analysis, etc.
In the ForLab project a new portable system combining LIBS and Raman analysis capability has been be designed and developed with the following characteristics:
Figure 3. LIBS-Raman system developed in ForLab
• Two modes of operation as table-top equipment or as backpack to be able to access and analyze samples that are difficult to reach
• Capability to detect different types of explosives directly on different surfaces or using a swab
• Analysis time, including automated response in less than one minute
• There is no contamination of the detector and therefore continuous operation is possible
• Designed to avoid cross contamination between samples.
• Capable of automated detection of particles of 90 ng of explosive with LIBS and particles with diameter of 300 µm with Raman
• Easy to update Raman substance database for identification
• Capability of detection of gunshot residues (GSR) on standard sampling kits and other surfaces, providing a reliable result in less than one minute.
During the final tests of the project the LIBS system achieved outstanding results being able to differentiate samples containing residues of the explosives from samples not containing residues (or in a quantity below the limit of detection of the system) in all the three scenarios.
B 1.2.4 NLJD for detection of electronic debris
The foundation technology (NLJD) was previously developed to detect small electronic components used in eavesdropping systems and electronic bugs and has been well known for over 30 years. This technology is based on the intrinsic property of non linear junctions such as those contained in semi-conductors, to radiate different harmonics, when radiated at a given frequency. It provides the ability to detect electronics, even if switched off or in sleeping mode.
The electronic components are detected in real time by measurement of the 2nd and the 3rd harmonic levels. Localization of electronic debris will be obtained using a man-portable NLJD detector. When the operator detects an electronic device, he pushes a button to inform the Command and Control Center that the current NLJD detector position within the 3D scene corresponds to the position of an electronic device. The position of the NLJD detector will be determined thanks a 3D scene reconstruction performed by a dedicated platform implemented with a LIDAR (Astrium’s contribution to the Forlab system). This platform is man-portable and has to be moved to different positions so that the LIDAR measurements can be combined to build up the 3D scene.
The precise position of the NLJD detector into the 3D scene, taking into account the sweeping movement of the operator scrutinizing the area will be obtained by a Kinect camera.
Figure 4 The full system with the NLJD scanner
B 1.2.5 3D Scene modelling for forensics
The start of any forensic investigation must begin with the collection, review, and analysis of evidence. The better the quality of evidence, the better the analysis and likelihood of solving the crime is.
Although photogrammetric techniques using cameras, tripods are regularly used, the trend today is to use directly 3D laser Scanning technologies. It is an ever growing and useful application of laser based measurement technologies in fighting crimes and reconstructing events.
Often referred to as High Definition Surveying (HDS), 3D scanning became popular in the late 1990s. Some scanning technologies are based on optical methods whereby photographs are used to collect and match points in corresponding photographs (i.e. Photogrammetry and stereo matching) while the lesser known CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) allow the interior structures of objects to be "scanned" and examined.
The more common 3D Lidar (Light Detection and Ranging) scanners emit a beam of light and measure the part of the beam that is reflected back to the instrument. These are the most common types of scanners being able to collect and preserve data from very large crime scenes.
The two most common types of Lidar scanners are “phase-based” and “pulse-based” scanners, which refer to the method for determining the distance to any surface that has been scanned.
Compared to classical approaches HSL techniques, allow acquiring data in detail without predetermining what is and what is not evidence. It allows scanning organic shapes and highly curved surfaces that would otherwise be difficult to measure (ex of bloodstains evidence spanning over several surfaces, furniture, and walls).It can physically reach point of measurement without touching, deteriorating or contaminating it. Most measurements can be done from a safe distance but equipment operators and the general public should be at safe eyes distances. real-time visual feedback as the points are being captured, are possible making the scene immediately available for review and analyzed at the scene. Areas lacking detail can then be re-scanned while other areas that did not scan properly can be addressed.
However, level of Required Accuracy depends on the practical range in given conditions. Some reflective surfaces, dark/highly absorptive surfaces or oblique surfaces may not provide any or good data. Registration errors, when combining several scans in one large point cloud, need to be considered.
There is now a migration towards photorealistic 3D environments where millions of points can be measured and analyzed. 3D scanning is still relatively new and costly so that other capable technologies such as photogrammetry, should be also considered as well as combination of the two.
The greatest area of development will be in the available software tools for the analysis and visualization of scan data. Some of these tools apply to bullet trajectories, bloodstain pattern analysis, and even "image projection" where a suspect's height can be estimated from security videos. Once the analysis is complete, the results can be immediately displayed as a 3D animation or through an interactive viewer for court presentations. Eventually, the benefits of 3D scanners and related analysis tools will become more familiar to law enforcement agencies, attorneys, and jurors.
The ForLab project developed the capability to produce 3D models directly on the field in an iterative way.
Figure 5. 3D modelling tool developed in ForLab
The Innovations of the approach are:
• Acquiring rapidly 3D models and not simply 3D surfaces or 3D geometric points, with the challenge of processing the data directly on the spot in order to deliver near real time data to the Command & Control Centre.
• Acquiring not only geometry and radiometry, but also semantics and eventually topology
• Allow easy integration (geo-localisation) of any complementary detections and measures without needing specific set-up on the scene (position reference patterns...).
• Allow rapid deployment through acquisition planning on the spot and rapid set up: typical deployment timing would be around 10 minutes, with acquisition lasting a few minutes, and the redeployment taking a few extra minutes.
• Specific tool to optimise the acquisition process to limit the number of acquisitions while preserving minimum occlusions.
• Ability to produce even from data with occlusions, complete 3D model with occluded areas interpreted: even if we do not see the floor behind the chair, or the wall behind the car, we can rebuild the floor plane or the wall plane to produce a complete 3D model.
B1.2.6 Sample position reference indoor and outdoor
Autonomous tracking and localization approaches that do not rely on GPS are becoming very attractive the last years especially for indoor navigation. The main characteristics of autonomous tracking systems, are deployment time, accuracy, portability and reliability. Moreover these systems ought to be pervasive and require zero interaction with the users.
Depending on the target application different requirements for each characteristic are needed, and as a result, radically different systems can be developed that depend on different working principles and are of different cost. The core design choice that serves different requirements, is the point –to-point distance measurement. This can take place by inferring distance, using physics laws, from measurable natural phenomena like sound time-of-flight, electromagnetic signal time-of-flight, electromagnetic signal angle of arrival, electromagnetic signal strength. Depending on the physics formulas that associate the aforementioned measures with distance, the achievable level of accuracy is determined. For instance, time-of-flight can be easily associated with distance because speed of sound and speed of light are well measured, stable and a simple formula associates them with distance. On the other hand, signal strength depends on a variety of factors that is very difficult or impossible to measure, and the distance association approach is to take indicative measurements and implement curve-fitting maths. The latter approach radically affects the deployment time because it requires a training stage, where the user needs an additional different method to measure the distance in order to train the system and accuracy can vary at runtime because several factor can affect received signal strength. But, there is no free lunch, what is straightforward to compute using robust physics formulas is very hard to measure in practice. More specifically, the light speed of electromagnetic signal requires clocks with picoseconds accuracy so that reasonable accuracy (10cm) is achievable. Current state of the art processors cannot compete with required performance, so application specific hardware accelerators are required to implement baseband processing which along with very sophisticated algorithms result in very expensive devices. In ForLab UTH implemented 2 approaches for the measurement of distance between 2 points:
Option 1: Received Signal strength of 802.11n (CSI metric) for low-end devices
Option 2: Signal Time of Flight using Ultra wideband(UWB) which was used in the final system and demonstrated in Linares.
These diverse systems achieved the following performance for distance measurement:
Main specifications of option 1 RSS:
• deployment needs training (20 min)
• 60 meters is the maximum distance that can be measured between 2 points.
• Accuracy of 1-2 meters
• Very cheap (30 € per device)
Main specifications of option 2 ToF:
• deployment does not need training
• 120 meters is the maximum distance
• Accuracy of 10 cm
• Relatively expensive (600€ per device)
Another important subsystem of the localization module is the tracking solver which uses distances and known anchor Cartesian coordinates to provide the Cartesian coordinates of a point on any mapping system. The main job of this module is to solve a system of non-linear equations that associate the given point with the known anchors. The math are standard and based on Non-linear least square solving approaches which come in several flavours. UTH thoroughly explored several localization areas solution space using different solving algorithms to determine their performance and appropriateness for the specific type of problem. Moreover, they have developed a novel methodology to detect a faulty distance measurement when the number of anchors is more than 3 and exclude the faulty measurement for the calculations. The approach is based on solving all possible combinations in pairs of 3 and inspect the convergence effort which indicates when a problematic measurement affects the solution region. The specific algorithm detects faulty measurements in 90% of the cases and is in the core of a patent application.
The final ForLab system field requirements lead to the adoption of option 2 for distance measurement. More specifically, the final system specifications are as follows:
• Position Accuracy 10cm
• Automated alignment with a any coordinates reference
• Minimum deployment requires 4 anchors (arbitrary placement)
• Minimum coverage 10.000 square meters (100m x 100m area)
• Response time depends on a number of concurrent devices.
• Each device introduces a 20msec delay for (1 second for 5 devices)
• 8 hours non-stop operation on batteries
• Weight of a ruggedized full set with batteries, 4 anchors and 4 location devices is 5kg.
• Prototype cost of a full set 12.000€ (8 devices).
• Average deployment time 15 minutes (needs trained personnel)
• Proven field operation (under rain)
Figure 6. Anchor points and positioning module in the office scenario
B1.2.7 Command and control centre (Knowledge Management)
“Command and Control” addresses the arrangement of personnel, equipment, communications, facilities, and procedures employed by a commander or a command center in planning, directing, coordinating, and controlling resources and operations in the accomplishment of a mission.
“Operations” deals with the way the operational process and procedures are conducted by users and deciders using many data sources to support decisions and to optimize results.
“Communications” looks into the way the data and commands are shared and transmitted among many sources and users linked together to support decisions.
“Knowledge Management” consists of:
• Sensor fusion for prediction and estimate of system internal states by optimal stochastic filtering of system and sensor dynamic models with real measurements, and
• Data integration, management, merging, analysis and exploitation of the multiple sources of data to support decisions and how deciders can interact with the data.
Command and Control Center
A Command and Control Center is typically a secure facility that operates as dispatch, surveillance monitoring, alarm and coordination office center. A command and control center that is used in a deployed location is usually called a command post.
Command and control functions are performed through an arrangement of personnel, equipment, communications, facilities, and procedures employed by a commander in planning, directing, coordinating, and controlling operations in the accomplishment of a mission.
Commanding officers are assisted by specialized staff officers and personnel. These staff provides a bi-directional flow of accurate, timely information between a commanding officer and subordinate field teams, which by category represents information on which command decisions are based.
The key application is that of decisions that effectively manage resources. While information flow toward the commander is a priority, information that is useful or contingent in nature is communicated forward to lower staffs and field teams.
The ForLab project developed plug-ins to:
• Enable forensic investigation professionals shall be guided through the scene.
• Enable presenting an overview of the scene that the professionals are involved in.
• Enable presenting possible evolutions of the situation based on the collected data.
Using visual navigation for after action analysis shall help:
• Professionals to perform more detailed debriefing of data and procedures leading to improved future operations.
• Better prevention and mitigation of risks.
The main novelties of the ForLab project with respect to command and control centre will be to:
• Bring state of the art decision making command and control techniques using:
o Dynamic modeling of data collection sensors supporting command and control centre
o Explore and model detailed application requirements that shall drive technology development
• Bring into the system data and information management for collected data reduction, browsing and rapid access.
• Bring system indexing of large scale distributed data
Data Integration and analysis
A data system must also be capable of integration, management, merging, analysis and exploitation of data, which in turn can be very complex due to the multiple nature of the different data and their sources. Besides, conflicts and multiple-access are also very common in the operational scenario due to wrong data, multiple sources and source conflicts in redundant data.
Data fusion is composed of a set of subtasks that filter, transform, reorganize and optimize data. Common Data fusion operators are: Minimum union, Full Disjunction, Complement Union, Merge and Priority Merge, Match Join and Group and aggregation
Although the engineering literature is replete with examples of how data fusion techniques are being applied in military and industry projects, they are just now beginning to be applied to many kinds of projects. Their integration is a challenge in order not only to optimize the data produced by them, but to improve their own work process.
The main novelties of the ForLab project with respect to data fusion, integration and analysis will be to address the:
• Server
o AMQP message broker (RabbitMQ)
o Key-based distributed data store (RiakCS)
o Application modules coordinated through AMQP
o REST service
• Client
o Eclipse RCP based product
o 3D renderer and scene management
o Graphing, multi-windowing, cross-platform
• Capabilities:
o 3D scene with in-situ evidences and aerial view
o Data visualizations for each sensor
o Rich client and mobile client
• Use of data:
o Replay investigations
o Simulated investigations
• Integration:
o Standards based messaging and plug-in architectures
o Distributed processing and analysis
Figure 7. Screenshot of the C&CC. Metro station scenario
Potential Impact:
As defined in the work programme, the expected impact of the topic SEC-2011.1.3-2 is the following
“To help law enforcement agencies in their analysis and to deal with explosive events in a common European approach”
ForLab has delivered a novel systematic methodology for optimizing the process of evidence collection and analysis. We aspire that our approach shall maximize the speed, reliability and accuracy of the process and ultimately make a significant step forward in the battle against terrorism reducing the time needed to identify the authors of the attack and increasing the effectiveness of the response of the Security Forces.
Today a new concept on the fight against the terrorism is widely accepted. This concept is based in the idea that the explosion of and IED is the final step in a chain of events driving any IED based attack:
1. The attack must be planned and financed
2. The people to perform the attack must be recruited
3. The IED must be prepared, a supply chain is necessary to support the construction of the IED
4. The IED must be emplaced
5. Finally the IED must be detonated at the desired time.
As we approach the step five, the chances to prevent the attack are reduced. Every time that there is an explosion of an IED it is of vital importance to retrieve as much information as possible about the inspirers of the attack, the materials used for the preparation of the device, the specific characteristics of the attack (emplacement, amount of explosive, etc) any clue that may reveal information about the previous steps that may lead to disrupt the chain of events of the next terrorist attack.
The time of reaction is a critical aspect of the fight against terror, the sooner the security forces can gain the knowledge about the author, the greater the chances of success. The information retrieved from the scene must be ready to be shared with Security Forces of friendly countries so that the terrorist and their support chain can be pursued beyond the border of the country suffering the attack.
The objective of the ForLab project is to reduce the time of response of the Security Forces by improving the existing capabilities for the analysis of the post-blast scene of an IED attack. This objective can be achieved based on the following aspects:
• Providing the investigations with tools to help on the localization of potential evidences
• Providing the investigators with tools for the objective evaluation of the quality of the evidences collected.
• Making information available immediately and in comprehensive way in the Command and control centre, making possible information sharing with other security body.
• Allowing expert investigators to guide less experienced investigators on the field form the remote Command and control centre.
• Proposing a new concept to approach the investigation of the scene based in full availability of the information gathered from the scene in real time to drive the investigation, and combine this information with the available information from previous events of similar characteristics. The success of this approach will be multiplied the proposed solution matches the procedures in use by Security Forces of different countries of Europe.
Focussed development of existing technologies from different countries in Europe to help Security Forces in the selection and localization of the evidences and the coalition of Security Forces from several countries with industry is only possible in the context of a European programme. ForLab is a good example o this cooperation giving as results some tools highly tailored to the common need of European security forces.
In particular, the achievements of the project in detection of traces of explosives will have very positive impact on the work of the laboratories of analysis in the investigation of a post-blast event:
- on the economic aspect : to be able to discriminate on the field between the elements and to select some of them more relevant, allows more effectiveness but also avoid to make a great number of expensive (and useless) analysis in laboratories thus reducing the costs and saving valuable time.
- Although all the results get on the field must to be confirm by a police (or a specialised) laboratory: the "analytical-field-discrimination" allows to concentrate the first effort on some relevant samples which gave positive answer (about traces of explosives) on the field.
If a first and relevant selection of the elements is made on the field, it allows to avoid the "jam effect" for the forensics laboratories. Indeed, if too much samples arrived at the same time to the labs with no priority given, we have a risk to "paralyse" the system, particularly in case of multiple attacks
Although ForLab was very focussed on the post-blast scene, during the execution of the project we have seen that most of the technologies and concept developed in ForLab can be applied to other crime scenes. Making a exhaustive description of the possible application of ForLab it is out of the scope and extension of this document, but we can just outline a few of those applications.
• The concept of making the electronic report on the collected evidences is exportable to virtually any crime scene.
• Te concept of creating the 3D model and 2D map of the scene and place the evidences on the map is already used in different crime scenes, but ForLab allows doing it in real time
• Getting the position of samples in an automated way (even not having the 3D model) will save a valuable of time in investigations with huge amount of evidences
• The LIBS and Raman systems have the capability to detect (or identify) virtually any substance, and the concept of a portable and reliable equipment can be exported to other crime scenes involving, to mention just a few, drugs, gunshot residues, chemical agents...
• In a similar way the LIF technology is applicable not only to the detection of polymeric debris, and the concept of analysing a an area from a standoff distance can be exported to other scenarios
• The command and control centre has been designed as an open architecture, making easy the integration of new tools that the security forces may be using in the future.
In particular, we must highlight that as a result of the project, two companies (Airbus Defence and Space and Indra) have ongoing projects to bring to the market a 3D modelling tool for forensic applications and a portable LIBS system for forensic applications with expectative to get them available within the next year.
Contact data:
Indra Sistemas S.A
F. Javier Hernández
Tlf: +34 987849888
e-mail: fjhernandez@indra.es
University of Thessaly, Greece
Dr. Dimitris Syrivelis
Tlf: +30 6944318012
e-mail: jsyr@inf.uth.gr
