Final Report Summary - BASYLIS (moBile, Autonomous and affordable SYstem to increase safety in Large unpredIctable environmentS) Executive Summary: The BASYLIS project was defined to contribute to increase the security of the European citizens by the developments of an adaptable and affordable system for temporal or permanent protection of facilities, perimeters and people using the combination of multiple technologies. Figure 1. Example of a critical infrastructure. (Can be seen in attach PDF)BASYLIS system consists of the following modules: - Radar- Ladar- UGS Acoustic (UNAVE)- COST Integration Board- UGS Metal- Bracelets and panic buttons- Seismic UGS (optimization)- Video Intelligent- Behavioural AnalysisCivil installations such as power plants are often located in wide and remote areas. In the future, the number of small distributed facilities will increase as a direct result of new European environmental policies aimed at increasing societies´ resilience to climate change. However, the protection of fragmented assets will be difficult to achieve and will require portable security systems that are affordable to those in charge of their management. The BASYLIS project aims to address these issues by developing a low-cost smart sensing platform that can automatically and effectively detect a range of security threats in complex environments. The principal obstacles to early threat detection in wide areas are of two types: functional and ethical. Both problems are exacerbated when either the installations or the environments are dynamic. Potential solutions are unaffordable to most of the potential users.The BASYLIS system is capable of detecting a wide range of security threats. Sensors within BASYLIS can sense different types of signals: radio, magnetic, seismic, acoustic and optical waves, as well as images via intelligent video.The BASYLIS system has been developed oriented to a high performance and usability. The engagement of end users in the project for the specification and validation of the design has been considered from the start of the project, ensuring that the design of the final system meets their needs.Project Context and Objectives:Context The lack of security in refugee camps is a fact, especially the critical suppliers for the population. In the last year 3.048 United Nations (UN) workers from 48 different countries died during peace missions. This is the worst balance in the 61 years of history of UN, which accumulates more than 2.500 casualties. Actually United Nations has 23.000 civil workers in 18 peace missions. This number does not include others Non Governmental Organization (NGO) that works in refugee camps too, including the Red Cross, Caritas etc. Security departments and forces demand new affordable system capable of detecting wide range of threats in extend and heterogeneous areas. In response, the BASYLIS project has developed a security solution that will combine RADAR, LADAR, acoustic vector sensors, unattended ground sensors, bracelets for personal security and video Intelligence for alarm recognition and classification. BASYLIS project will also process behavioral analysis tools in combination with the above mentioned sensors and with social science.The BASYLIS system is an adaptable system, capable of covering a wide spectrum of threats and able to perform in many large unpredictable environments. The project aims optimizing the sensors use for each situation and reducing the cost.ObjectivesThe objectives defined in the BASYLIS project are the following:1. New sensors development (radar, ladar, UNAVE, UGS, bracelets, intelligent video), focusing on their potential cross-integration with the others, and devoted to the infrastructure protection.2. To develop the main integration software and behavioural analysis enabling the integration of the sensors and the behavioural processing of the data.3. To define real scenarios with the active involvement of the end users in the project.4. To integrate all the software and hardware developments in the selected demo scenario.5. System test on real scenarios. 6. System demonstration to end users and technology stakeholders. 7. Elaboration of the exploitation plan of the project results. Project Results:1) Work package 1: Capabilities road mapThe aim of this work package has been to study and define different scenarios, system requirements and technologies used for protection and surveillance of certain infrastructures. Within the scope of this work package, according to the features of each threat and the scenario, a matrix has been created where the different threats are identified and the technological solutions that are addressed to that specific threat. Thanks to this study, the grade in which each technology could be improved in the project framework has been defined. The threats have been shorted in four groups:- Actions related to arms or to insurgents.- Gender violence.- Other common threats.- Attacks or damages to UN staffs.Each threat needs different type of protection that applies to different areas of the facility:- Perimeter security- Facility security- Social security- Personal securityAfter finding the threats and areas to protect, a matrix with the solutions was created:Protection system Protection addressed to...UGS Metal sensor Inner areas protection. Detection of people transporting weapons.UGS Seismic Wide areas protection: the perimeter of the protected area besides roads and high risk assets. Inner areas protection.Panic bracelets and buttons Personal security. High risk areas, such as places with lack of illumination. Staff protection.Acoustic Vector Sensors Inner areas protection. Radar Wide areas protection. Perimeter and facility security. Perfect for behaviour analysis.Ladar Temporal settlements: supply convoys. Perimeter security: intrusion detection, such as cars, people etc. Video analysis Alarm verification.Behavioural analysis Strange behaviours detection.2) Work Package 2: Perimeter Security The objective of this work-package was to study and develop low cost sensors to detect threats and to protect open areas at long distance. This task was divided into seven particular objectives:1. Development of new transceiver module. TERMA has developed a new module to reduce the cost of the equipment, enhance performance and increase reliability. The module operates in the Ku band.2. Mechanical Design. TERMA has developed a new mechanical design to improve of the mechanics of the radar to make it more ruggedized, without compromising size, weight and power consumption. The mechanics is redesigned so only the antenna rotates.3. Hardware Integration. TERMA has conducted integration of the hardware works of the other consortium partners.4. New antenna design. UFL has developed a new antenna for the radar and produced a prototype. The antenna is of flat design of the type RLSA Symmetric Beam Antenna5. Development of Supervision Functionality. NTGS has developed a new board for supervision functionality of voltages and temperature in the radar mechanical unit.6. Develop high resolution radar processing and post processing. UPM has defined and developed a new set of waveforms to improve the radar range. INDRA has developed new signal processing capabilities. 7. Development of LADAR Multiple Elevation development. Improvement of the Ladar functionality to provide multiple elevations, reduction of false alarms, and mechanical improvements to provide protection against environmental and transportation impacts.Result of design of new Transceiver ModuleThe new Transceiver Module is part of a new design for complete GSR radar by TERMA. The aim was to design a robust module which integrates easily into the new design. It requires only convectional cooling from the surrounding air. The module is the basis for the 8 Watt Transceiver module in the TERMA GSR. The mechanical layout and photo of internal electronics circuits is shown below.Figure 2 Transceiver Module electronic circuitry (Can be seen in attach PDF)Result of new mechanical designThe mechanical design is based on stationary electronics within a heat sink rimmed enclosure and a radome with a rotating antenna driven by a brushless motor with a hollow shaft encoder. This design is robust and withstands required environmental specifications.The design includes passive cooling of the internal electronics via convection of heat from the electronics heat sinks to the circulating air. A prototype has been produced for Basylis hardware integration and verification trials. This prototype was configured for the test and integration, including the amplifier, power supplies with power filtering, motor controller, circulator, brushless motor, encoder, and integration of new antenna. A photo of the prototype is shown in Figure 3.Figure 3 New mechanical unit. (Can be seen in attach PDF)Result of antenna designRLSA design. The antenna design includes concentric slot rings RLSA based on a feeding travelling wave guide. It makes use of a reliable dielectric material. It is a Rohacell WF110. The antenna geometry is characterized by:• maximum slotted area <38cm• maximum antenna radial dimension 40cm• thickness 1cm (without the assembly bracket)• dielectric filler of the RLSA: 6.6mm thick Rohacell WF110A prototype has been produced for measurements in antenna test site and for use at Basylis field trials, Figure 4(a).Figure 4 The antenna LPRLSA prototype (a) and reflectarray prototype with cosecant square pattern (b) (Can be seen in attach PDF)Reflectarray design. Two reflectarray prototypes, which radiate a cosecant square pattern, were designed and realized. The planar dimensions of the first and second prototype are 38.60cm x 22.81cm and 36.82cm x 21.04cm respectively. For the first prototype the pattern exhibits a 3dB angle of about 3° in the plane φ = 0° and of about 13° in the plane φ = 90°. The estimated maximum gain is 28.63dB. For the second prototype the pattern exhibits a 3dB angle of about 3° in the plane φ = 0° and of about 11° in the plane φ = 90°. The estimated maximum gain is 29.30dB. Figure 4(b) shows the two realized prototypes and some of their manufacturing details. Result of development of supervision functionalityThe Supervision Board consists of both hardware and software application developed by NTGS. It is able to monitor the temperature and voltages inside the radar and to provide a warning or error message when the measured values are not within the defined thresholds.Figure 5. Supervision Board, with detail inside. (Can be seen in attach PDF)Result of development of high resolution radar process• The maximum range of the radar has been doubled without changing the hardware by implementing new waveforms based on Barker codes.• The High Resolution Ranging technique based on Step-Frequency processing has achieved a 2 meter resolution.• The clutter map is computed during the radar set-up and updated periodically.• A CA-CFAR technique with configurable parameters has been developed. • The plot integration process is performed as blob detection in the range-azimuth-Doppler matrix. The blob detection is implemented using the connected-component labelling technique with parameter tolerance.• Track classification with configurable tolerance parameters has been developed.. Result of LADAR Multiple Elevation developmentBasylis and NTGS individual trials have shown that the Ladar system performance is successful operationally with a detection range up to 500 m. The advantage of the multiple elevation of the ladar, is not only for mountainous areas in order to avoid blind angles, but also in uneven grounds. Thus, ladar operation in multiple elevations improves detection capability. The mechanical elevation movement was implemented by means of a stepper linear motor, able to elevate or lower the laser beam up/down to five different positions. The normal mode is 360 turn at 10 revolutions per second. This feature can be modified because of the parametric software to adapt its accuracy to the terrain conditions.Because of its light weight and small size, it can be rapidly and easily deployed in any environment. Besides, it can be powered by battery, solar panel or electric supply. The averaged consumption is low, less than 0.8 A for a supplied voltage 12 - 14 V DC.Ethernet or wireless communications could be used. Furthermore, the deployment of several ladar with a control centre would behave as a surveillance barrier.Figure 6. Detail of the LADAR telemeter position, elevated, during the initial trials. (Can be seen in attach PDF)Figure 7. Shot of Ladar software details: group factor, velocity, elevation, etc. (Can be seen in attach PDF)3) Work Package 3: Inner Areas Protection Acoustic Vector SensorExisting Stat of the ArtTesting of the AVS system is done with a earlier version sensor which supports the same functions as this UGS system, one of the noticeable difference is an internal PC for easy and quick testing (of software, hardware and accuracy) which now has been replaced by a dedicated DSP reducing the power consumption. In Figure 6 the system is as it is at the moment. Figure 6. UGS (Can be seen in attach PDF)The AVS sensors are very small (26cm diameter, less than 15cm height), lightweight (1.7kg) and consume including data transmission low power (2W normal operation with Xbee transmitter, 350mW sleep mode). Due to the unique and patented Acoustic Vector Sensor (AVS) technology, the AVS is capable of localizing acoustic events in the complete acoustic bandwidth. For the AVS system this is typically limited to between 10Hz and 10kHz.The AVS system consists of four parts: a mounting with windscreen, a battery pack, an antenna and a solar panel see Figure 7. Figure 7. The block diagram of a UGS and extra equipment (Can be seen in attach PDF)The electronics that operate the AVS System are placed in the elevated watertight box. It is therefore well protected against physical damage. Under the elevated box there is an inner protection layer. This layer is used to remove wind induced air turbulences and protect the AVS sensors from dust. The outer protection layer is a composite layered material that repels water and cancels wind noise.The electronics consist of a preamplifier for the AVS, a digital signal processor and X-Bee transmitter. The AVS is mounted under the elevated watertight box in a watertight way. Small coaxial connectors are used for the external power and the X-Bee antenna. There are two external connections on the AVS: the 12-14VDC power input and the antenna output. In this version the transmitter is built in.The AVS are positioned in the field and communicate wirelessly to the Mainstation Control Post (MSCP). The main purpose of the AVS is to detect, classify and determine the direction of arrival (DOA) of an acoustic event and to communicate this to the MSCP. The main purpose of the MSCP is to collect the reports, determine which report is associated with what event, calculate locations of the events, visualize the events on a graphical user interface, and to communicate the localized weapon positions to a separate Human Management Interface (HMI). All reports are stored and can be replayed in the MSCP display. The user interface of the MSCP is Windows based and highly flexible. There are several modes of operation that allows the software to be used by people with different skills.In this project the localization of small arms fire is included in the software. This means that the location of a shooter can be calculated with a single sensor if the shock wave and muzzle blast are detected. These algorithms are ready. However in this trial an alarm pistol was used as sound source. Without the supersonic bullet the shock wave is not available and the localization of the shooter with a single sensor is not possible. In this demonstration the shooter location is determined with the use of multiple sensors.PerformanceAt the official trial day -March 2013- approximately 100 shots (alarm pistol - bullets: 9mm blanks) were localized in 26 trials. In most cases, all shots were localized consistently, even though the shots were fired closely after another (within 1 second).An example of the results of a trial is given in Figure 9. Two shots were fired at each shooting position and all shots were localized. The localizations given by the Acoustic Vector Sensor system are reasonable, the distance is between 3 and 8 m.The accuracy of the system depends strongly on the accuracy of the GPS measurement for the sensor positioning. In this case the inaccuracy of the GPS tool used to position the sensors is approximately 3 meters for the sensors and shot positions. Considering the distance (20m) between the sensors and the accuracy of localization, this error in GPS position is undesirably large.Figure 9. Localizations by the Acoustic Vector Sensors during a trial where two shots are tired at each shooting position. (Can be seen in attach PDF)At all positions, the false alarm rate of the system was very low. Passing cars were observed by individual sensors, but produced no (false) localizations. ConclusionThe AVS system is capable in detecting gunshots without any false alarm. The trials of October have (relative) better results comparing to the trials of March. This because of relative error the GPS is generating (accuracy of <2 meter). With a smaller trial area the GPS accuracy is getting more important. The results of the AVS System is very accurate and within the scope of the project. The hardware is showing no errors and is intensively tested to provide a high quality system. The software is well integrated with the defined UBDF protocol for sending information to the HMI. COTS Integration BoardIn order to reassure the success of Basylis solution both in efficiency and low cost terms, the seismic and/or metallic sensor will be integrated on a board. This COTS (Commercial-Off-The-Shelf) board has become a key element in Basylis development.As Basylis frequencies ranges from 2.4GHz to 160MHz, two different modules (COTS boards) with two different transceivers needed to be developed to comply with the radiofrequency sensor requirements:The microcontroller MSP430 used in Basylis, Texas Instruments Proprietary solution, is an ultra-low-power 16-bit RISC mixed-signal processor.a. MSP430 + transceiver cc2500The CC2500 is a 2.4 GHz transceiver for very low-power wireless applications. The circuit is intended for short range device frequency band.b. MSP430 + transceiver cc1120The CC1120 is a device for short range frequency bands at 164-192 MHz, 410-480 MHz and 820-960 MHz.Power consumption is one of the most critical requirements for Basylis solution. This microcontroller is the best microcontroller for battery-powered applications for underground sensors. In addition, it has enabled to design interfaces to analogue signals, sensors and additional Basylis components ensuring low power consumption. Communication protocolAs far as the communication system of UGS is concerned SimpliciTI (also a Texas Instrument) will be applied as it is a low cost and low power wireless capability design element.SimpliciTI operates on 802.15.4 target or non 802.15.4 targets, what is ideal for the two boards designed (microcontroller MSP430 + transceiver CC2500 and transceiver CC1120). It requires a very low HW abstraction, and no need of driver support.(Image Can be seen in attach PDF)As the communications will be integrated in the sensors, SimpliciTI is the best standard to be applied in comparison with ZigBee because: It provides lower transmission time and computing power due to smaller header used by data packet. Of the flexibility of operation frequencies, covering ISM international band from 166MHz to 2.4GHz. Master-Slave communication.Metal UGS for Weapon DetectionTheir core capabilities will be avoiding the proliferation of small arms as well as carrying and smuggling weapons. Additionally, these Underground Sensors will prevent the presence of insurgency, rebels or any other type of intrusion for whatever reasons: raids, looting, vandalism, demonstrations... Not only will these sensors protect the refugees accommodated in the camp but also UN staff and facilities within the camp.Metal and seismic sensors to be used are underground: this is the most relevant characteristic, since it will prevent the previous detection of them -they can be buried up (down) to 10 cm. This also reduces sabotage likelihood. Additional features are: - Autonomous - Low power consumption (battery life: 6 months)- Seismic sensor- Metallic sensor- Reduced size (10 x 10 x 10 cm)- Weight lower than 500 g- Can be buried down to 50 cm max.- Low cost- Communication system integrated- 5 m range- 10000 h MBTFTarget Classification based on Frequency AnalysisThree types of elements acting in the system will be differentiated: human beings, animals and vehicles. The main difference between persons and animals is that biped beings have less step cadence than quadrupeds, but a properly defined frequency. On the other hand, vehicles frequency-spectrum centres these components near 0 Hz.4) Work Package 4: Personal Security The development of personal protection was based on two premises: affordability and portability. Thus, a COTS bracelet was complemented with a portable GPS unit. Figure 27. Bracelet and GPS Unit. (Can be seen in attach PDF)The bracelet, eZ430-Chronos, is a wireless development system with CC430F6137 microcontroller system-on-chip and integrated RF transceiver, working at 868MHz. this frequency is within Basylis frequency range. Besides, it runs SimpliciTI, communication protocol used in Basylis.Affordability is also guaranteed as it is powered by a lithium coin cell battery of 3V.The GPS Unit consists of Basylis COTs board with C CC430 and wireless transceiver inside. A SmartGPS was integrated to obtain Global Positioning System Fixed Data (GGA message):SiRF Star III, TTL level output serial port, WGS84, NMEA-0183 ASCII, 4800 bps UART.When the bracelet button is pushed, the GPS sends a message with alarm position: every 3 seconds for 5 minutes.Figure 28. Panic Buttons (Can be seen in attach PDF)This box, also developed by NTGS, features a Basylis COTS board with C CC430 and transceiver inside.It has been designed to be easily placed on any dangerous spot due to the small size.When pressing the two buttons simultaneously, it sends id_panic_buttons to the control centre via RF.Figure 29. Communications network for personal security devices (Can be seen in attach PDF)As described above, all the components of the personal security solution for Basylis supportt wireless communication. The Panic buttons, considered as nodes, as well as the bracelets, have Basylis COTS board with MSP430F537 and wireless transceiver inside, (868MHz). Also, all of them connect with a base (sub network 1). These bases connect with the control centre using Ethernet with twist wires. 5) Work Package 5: Automatic Alarm Verification The goal of the Automatic Alarm Verification functionality is to reduce the false alarm ratio. The Behavior Analysis System, Automatic Camera Positioning Software and Intelligent Video Analysis System are listening at the same time through common UDP Multicast group, as described in Figure 13 below:Figure 13. Module structure for target verification (Can be seen in attach PDF)When Behavior Analysis System needs camera verification, it sends a UBDF datagram to the group with an identifier that can be only read by the automatically positioning camera (the id will be configurable and the UBDF datagram from the BA module should have all the information required about the target, such as position and velocity). Once the Automatically Positioning Camera has finished the movement of the camera, it send another UBDF datagram to the multicast group with the same information but with another ID that only the Intelligent Video Image Analysis system can read. Therefore, the Intelligent Video Image Analysis System could start analyzing the image and therefore, it could send the information to the original Multicast group, where the rest of systems are also connected, like another sensor.Camera control and interface software management The system gets a sensor alarm as input and actuates the PTZ (pan/tilt/zoom) camera accordingly. The camera is positioned towards the intruder´s position given by the sensor. Eight different configurations were developed. Based on the installation environment and the operational requirements, the most suitable configuration can be used in every application. Configuration 1: The input data are the height of the target, as well as the coordinates of the camera and the target. This is used for observation of non-moving targets.Configuration 2: The input data are the position of the camera and target and the distance. The camera is turned to the direction of the target and zoomed on it to obtain images with optimal target height.Configuration 3: The input data are the position, distance and speed of the target. The camera modifies the zoom ratio according to the movement of the target.Configuration 4: The input data are the position, the distance and the speed of the target as well as the time. An estimate for the future position of the target can be calculated, and the camera can focus on the predicted position of the target.Configuration 5: The input data are the position, the distance and the speed of the target, time and the elevation of the target relative to the camera. The Tilt function turns the camera vertically to be able to focus on the target.Configuration 6: The input data are the position, the distance and the speed of the target, time and the elevation of the target relative to the camera. Based on a map or elevation plan, the system can calculate also the elevation of the target at a given time. The Tilt function changes the vertical angle of the camera according to the movement of the targetConfiguration 7: The input data are as in the previous configuration. Instead of just one, several cameras are used to track the target. The camera that is used for the classification is chosen by the position of the target at a given time. When the camera cannot see the target anymore due to obstacles, another camera is activated.Configuration 8: The input data are as in the previous configuration. Several cameras are used like in the previous configuration. Several targets are observed; each with the camera that has the best unobstructed view of the target and is closest to it.Intelligent Video Image Analysis SystemIn order to improve the performance and quality of the classification, the classification process was limited to objects that were previously detected by a motion detection algorithm. This decision was based on the presumption that the intruders approaching the refugee camp are moving towards it instead of just appearing in the surroundings, which allows them to be detected by means of motion detection. Therefore, the Task was divided in two subtasks: Motion Detection and Object Classification.Motion Detection AlgorithmBased on these parameters, Mirasys Ltd. developed the TR Algorithm, an efficient real-time motion detection method based on Local Binary Pattern (LBP) texture measure, morphological reconstruction and background modeling. Thanks to texture analysis, it is possible to detect motion even when the color of the pixels is very similar to the background because the neighborhoods of the pixels are taken into consideration. This applies also to classification tasks performed during the night, as thermal camera images operation on gray-scale basis. The functionalities and phases of the TR Motion Detection Algorithm reconstruction are showed in the images below:Figure 14. Phases of the TR Motion Detection Algorithm (Can be seen in attach PDF)Classification AlgorithmNext, Mirasys Ltd. developed a Classification algorithm that classifieds the detected objects as persons, vehicles or other objects (meaning that these most probably are not persons nor vehicles). The image analysis algorithm used in BASYLIS is based on Human Detection via Classification on Riemannian Manifolds. For BASYLIS project two classifiers were used: a car classifier and a human classifier. Car classifier uses the same training algorithm with human classifier. Car and Human detection has following confusion matrixes which are calculated using challenging Mirasys Car and Human dataset:Figure 15. Confusion matrixes for Car and Human detection (Can be seen in attach PDF)From matrix (A) it can be seen that there is total of 1402 cars. Classifier classifies 1323 of those correctly as a “Car”. Rest 79 cars are classified incorrectly as “Not a Car” but correctly “Not a Human”. Respectively there are 2816 humans and all of those are classified correctly “Not a Car” and 2586 are classified correctly “Human” and 230 are classified incorrectly as “Other”. Albeit with test set there is no confusion between Car and Human with real video feed some confusion have appeared.False positive ratios can be calculated from matrix (B) and (C). For human false positive ratio is 225/69775 = 0.0036 and for Car it is 1100/68900 = 0.0160. False positive ratio for car is expected to lower at least to same level with human after classifier includes all our improvements and corrections. Also human classifier is expected to perform even better after it is trained again with those improvements.When training both car and human classifiers emphasis has been given to keep the classes from being confused between each other. As a result if it is expected that object is either Car or Human, classifier can be tuned so that it has almost 100% accuracy. But this accuracy decrease with the function of how many search windows is needed and how many of them is neither Car nor Human.However, compared to other methods, the greatest difference is achieved by generating huge and representative training set. Usually car and human classifiers are trained using only few thousands positive samples, while the classification algorithm developed in the BASYLIS project was trained using a very large dataset of more than 25000 positive samples and 1500000 negative samples. Thus the resulting classifiers generalize well.System ImplementationThe kernel of the classification system is Mirasys Video Management System (VMS), which acquires the images from the camera(s), digitizes them, displays the images and stores them on a Hard Disk. The software modules containing the algorithms developed during BASYLIS project are integrated to the VMS.The Motion Detection Module containing the algorithm was integrated into the VMS. It receives the images from the VMS, analyses them and passes the results back to the VMS for further delivery to the Object Classification Module. The Object Classification Module is also integrated to the VMS. It receives the images and the motion detection data from the VMS, carries out the analysis and stores the classification results for each detected object into the VCA database of the VMS.The Classification Service acts as an interface sensor for communication with other parts of the BASYLIS system. When the camera is turned towards the detected objects, a message is sent to the video analytics system in the UBDF format. This message acts as a trigger to the classification service to retrieve the results from the data base and return them to the requesting unit.When a message with the classification command arrives in UBDF format, the service looks for the starting point of the search. It then starts to look for images with classification results. If it finds such an image, it sends the results of all classified objects in that image as separate messages; all of them with the same time stamp (time stamp of the image). If it does not find any classification results before reaching the end point of the search, it sends a message with target_class type ‘empty’.Mirasys Spotter is an advanced user interface for the VMS. For testing purposes, a plug-in software module was developed to display the findings of the Classification Module as notifications on the screen. The detected and classified objects are marked with icons showing their class (Person, Car, Other). These icons are colored according to the confidence factor of the classification (red-yellow-green). The BASYLIS Plug-in is presented in the images below:Figure 16. Car and Human detection and classification results displayed on Mirasys Spotter User Interface using the BASYLIS plug-in module (Can be seen in attach PDF)6) Work Package 6: Situation Awareness System Human Machine Interface (HMI)The Human Machine Interface (HMI) displays the information to the end user. The HMI system accepts two different types of input data, tracks and alarms. These two abstractions are equally represented by the UBDF message format, and allow sensors with different technologies to gather the information generated in a common format. The system has the four parts which are shown in Figure 17. Firstly, the sensors and their associated software components generate the input information for the in the form of UBDF´s messages. Secondly, the areas manager broadcast the received UBDF messages to all the active instances of the operator´s console. Thirdly, the operator´s console displays the data generated by the sensors on the map, as well as the location of sensors, security areas and icons. Finally, the notification server provides configuration support to the components.Figure 17. Systems global design (Can be seen in attach PDF)The areas manager compares tracks with security areas to detect intrusion. The areas manager does this comparison once for the whole system instead of letting each console to do it by them self. The notification server provides each client with system configuration parameters and updates the persistent data of every instance of the operator´s console when the administrator makes changes.The operator´s console allows two different profiles of use, as operator and as administrator. The console obtains configuration parameters of the system from the notification server, while the information about tracks and alarms is obtained from the areas manager.Figure 18. Operator´s console(Can be seen in attach PDF) Figure 19. a) track and b) intrusion in an alarm area. (Can be seen in attach PDF)SimulatorThe simulator simulates the basic behaviours of entities and sensing systems to reproduce the sequencing and timing of different sensing systems, investigating the properties of sensors, and to investigate scenarios which cannot be replicated using the actual hardware for time / logistical reasons.The simulator is configured using the same environment database as the multitracker. Two classes of agents – vehicles and people – are supported using an open source library (libpedsim) and a simple controller. Four types of sensors (RADAR, LADAR, Intelligent Video System, Acoustic Vector Sensor) are simulated. The simulations provide basic returns to illustrate properties such as noise and track breakage. Figure 20 shows a simulated output from the RADAR sensor.Figure 20: Example output of the simulator. The ground truth tracks are the solid lines. The circles denote radar reports, which are corrupted by noise and track breakages. Different colours indicate different track IDs. (Can be seen in attach PDF)MultiTrackerThe multitracker fuses the information from different sensing systems together to produce three advantages – to provide more descriptive tracks (by fusing attributes from the different sensing systems), to improve the overall coverage of the sensing system (by fusing the observations from all sensing systems), and to reduce errors in tracking both in terms of the number of false tracks and the accuracy of the trajectory of the track. The multitracker uses the generalization of a Track Segment Graph (TSG). A pose graph formulation of the fusion algorithms were used to support multi-rate, heterogeneous and out-of-sequence data.Sensing system drivers were implemented to handle the information from the LADAR, RADAR, Acoustic Vector Sensor, Intelligent Video System, Bracelets, Seismic UGS, Metallic UGS, and Buttons.The performance of the multitracker was examined over 25 different trials of varying levels of complexity. For scenarios which involve a single person, the multitracker has no difficulty. Figure 21 shows the output from one example. Here, the information from the different sensing systems has been combined together to produce longer tracks. Furthermore, each track incorporates additional state attribute information. When multiple people were being tracked, there were significant problems with unresolved targets. As a result, the output from the multitracker shows mixed behaviours, in which some tracks are well-defined, but some less clearly so. A further difficulty is that some sensing systems can be produce clutter tracks and false reports. Although various strategies were applied to identify clutter tracks, false positives were still periodically created.Figure 21: The fused trajectory from Trial 1.1. The diagram shows the raw sensor returns together with the filtered and smoothed estimate. The single entity track combines the different sensors together. This provides fuller coverage over the full sensing region, and annotates the tracks with additional information such as the gunshots. (Can be seen in attach PDF)Behaviour analysisThe behaviour analysis system monitors the behaviour of the entities to identify behaviours which are potentially problematic. These, in turn, are used to generate a set of high-priority warnings which appear on the HMI. The set of entities from the multitracker are passed to a set of behaviour rules. These rules use different criteria to determine the conditions under which alerts can arise. The alerts are then prioritized so that the highest alert level for any given entity is chosen and is dispatched over the UBDF network to the HMI. Alerts can be generated if a person is in a sensitive area for too long, if a person enters a sensitive area, if a shot is detected, or if the alarm button is pressed.In addition to the above behaviours, we also developed a classifier to test for following behaviours. Figure 22 shows the output from experimental exercises. Using data from 10 pairs of participants, we identified four behaviours (only one of which is problematic), and developed a Gaussian Process classifier which was able to classify following movement correctly 96% of the time.Figure 22: The second experimenal trial, and figures of the running-stop and closing behaviours. (Can be seen in attach PDF)7) Work Package 7: Trials, Demonstrations and Validation During the last months of the project, several field tests were carried out in order to test the correct performance of the sensors in a real environment. For that different intrusions where represented, of people and vehicles. The conditions set in these tests represent reality-based situations. - FACILITIES- All tests performed in a real environment were carried out in the facilities of Enagas, near Yela (Guadalajara, Spain). This installation gathered the characteristics ideal for the implementation of the system: the need to establish a security perimeter, intrusion possibility, possibility for placing sensors.Figure 23. Enagas facilites (Can be seen in attach PDF)- Testing activities performed in October 2012.The first global test carried out on the project was defined in the month of October 2012. Their goal was checked all the sensors developed, as well as collection of data for the training phase multitracker modules and Behavior Analysis.The sensors, radar, ladar, camera, seismic and acoustic, were tested in these tests and their data were sent to the UCL. Furthermore, and due to problems with bracelets and buttons in these facilities, the tests had to be performed in the facilities of Indra, sending the results to the UCL to the use of this information.Figure 24. Location of the sensors in the October trials. (Can be seen in attach PDF)The tests were performed during October:- A person walking- Two people walk 5 meters, one after another.- Two people walk 5 meters, next to each other- Two people walk 30 meters, one after another.- Two people walk 30 meters, next to each other- Vehicle- Testing activities performed in January 2013.The following tests carried out in the project were defined for some of all of the sensors that had submitted a minor problem or had not been tested in previous tests. It is for this reason the acoustic sensors were not required.Figure 25. Sensor Location (Can be seen in attach PDF)In order to collect as much information as possible and reflect the rest of members who did not participate in the tests, an internal document was created. It could distinguish two sections. The first one consisted of the collection of information about the individual operation of the sensors.Moreover, the second section was created to track the test defined in the test. These tests were classified as follows:- A person walking- Two people walk 5 meters, one after another.- Two people walk 5 meters, next to each other- Two people walk 30 meters, one after another.- Two people walk 30 meters, next to each other- Vehicle- A group of people- Two people walk separated by less than a meter- Four people walking on a nearby group- Four people walking on a closed group with a person toward and away from it.- Two people crossingFigure 26. Trials performed(Can be seen in attach PDF)The data collected from sensors was provided to UCL for its processing. - Testing activities performed in March 2013ObjectivesThe aim of these trials is to test the proper performance of the system as a whole. In this trial, all sensors developed in BASYLIS project were used: Radar, Ladar, Seismic UGS, Bracelets, Buttons, Metal UGS, Camera, Acoustic Vector Sensor and Intelligent Video. All the collected by sensors has been provided to UCL partner for Behavioural Analysis processing. Figure 27. Sensors location (Can be seen in attach PDF)In the figure above, we can see the sensors location in the test area:For these trials, the trials protocol has been updated in order to adapt it to UCL requirements with the purpose of capture data to Behaviour Analysis development. These data were collected by UCL’s logger and later were sent to UCL.On the other hand, the trials protocol was successfully passed, which consisted of the following tests:- One person tiptoeing, walking, running.- Two people walking separated by 5 meters (one next to other).- Two people walking separated by less than 1 meter.- Four people walking closely as a group.- Four people walking closely as a group with people leaving and joining the group.- Vehicle- Vehicle and people.- People crossing between them.In this test we could observe by means of the HMI and Simulator the real-time behaviour of the sensors. Both interfaces represented coherent results and run after aligning the sensors. Due to this reason, the final result was positive.- Testing activities performed in April 2013During the month of April, a pilot of the system was set up with radar, camera, multitracker and behavior in order to proof this part of the system. It was verified that the communication from the multitracker/behavior analysis to the automatic positioning and from the camera to the intelligent video and from the intelligent video to the behavior analysis was correct, the UBDF data format was correctly transferred.Potential Impact:Potential impact BASYLIS project was addressed to topic “SEC-2010.2.3-3 Automatic detection and recognition of threats to critical assets in large unpredictable environment”. BASYLIS system is the first perimeter security solution that is transportable, it has been developed focussing on the protection of refugees camps and critical infrastructure protection, such as reservoirs and electrical power plants. The challenge on the two facilities is two fold: In refugees camps can be over 10.000 people, are placed in places where are conflicts, these settlements are established in a hurry; usually there are not basic supplies such as electricity and communications. Concerning critical infrastructures, pipelines, electric gridlines etc. the wide extension to protect is the main challenge. Furthermore, it can be located in remote areas, such as deserts o forest areas. Critical infrastructures have the same difficulties as refugee’s camps in terms of access, energy supply and communications.BASYLIS project has achieved most of the objectives foreseen in the proposal preparation phase, which are very challenging and will enhance the competitiveness in the security technologies market: BASYLIS system is the first perimeter security solution that is transportable, can be deployed in a very short time, 2 hours. Adaptable to changing environments. High reliability of alarms thanks to the wide spectrum of sensors involved. Easy follow up of alarms, thanks to the fusion of the different alarms in a single one. BASYLIS is a scalable system, other sensors can be integrated into the system or can be excluded, depending on the needs of the area to be protected. The cost of the system has been reduced considerably, comparing to the solutions available in the market. The autonomy of the sensors in terms energy and communications has been increased. Furthermore, BASYLIS project and its results have been promoted during the whole project lifetime, in press media: there are twelve references. The project has been presented in several conferences, such as “EuCAP 2012 - 6th European Conference on Antennas and Propagation”, where the University of Florence presented the results and progress on the new antenna developed within the project framework. Thanks to the good results obtained during the project lifetime, the consortium partners decided to present the project in a stand in the Counter Terror Expo in London, which is one of the largest trade fairs in security in Europe. The three SME involved in BASYLIS project has developed new products, that very probably will be brought to the market soon: MicroflownAvisa, has developed the new acoustic vector sensors. Mirasys, has developed a fast and automatic alarm verification system that provides a new affordable solution to reduce the false alarm ratio in wide area protection systems. NTGS has developed a new ladar with elevation control. BASYLIS project has strengthened the leadership of European companies on security market related technologies. Dissemination activities Press releaseDuring the project numerous press releases where the Basylis project has been presented to both end users and any general public sector:ARTICLE TITLE NAME OF THE MEDIA COMPANY LINK TO THE ARTICLE DATE OF PUBLICATIONIndra desarrolla una solución de seguridad destinada a campos de refugiados elEconomista.es http://www.eleconomista.es/mercados-cotizaciones/noticias/3660829/01/12/Economia-Empresas-Indra-desarrolla-una-solucion-de-seguridad-destinada-a-campos-de-refugiados.html11/01/2012Indra desarrolla una solución de seguridad destinada a campos de refugiados infodefensa.com http://www.infodefensa.com/?noticia=indra-desarrolla-una-solucion-de-seguridad-para-campos-de-refugiados12/01/2012Indra desarrolla una solución de seguridad destinada a campos de refugiados Ateneadigital.es http://www.revistatenea.es/revistaatenea/revista/articulos/GestionNoticias_7063_ESP.asp11/01/2012Indra desarrolla una solución de seguridad destinada a campos de refugiados europapress.es http://www.europapress.es/nacional/noticia-rsc-indra-desarrolla-solucion-seguridad-destinada-campos-refugiados-20120111125109.html11/01/2012Indra desarrolla el proyecto Basylis para la seguridad integra de los campos de refugiados. diario_responsable http://www.diarioresponsable.com/innovacion/innovacion-social/15002-rse-indra-desarrolla-el-proyecto-basylis-para-la-seguridad-integral-de-los-campos-de-refugiados.html11/01/2012Indra desarrolla una solución de seguridad para refugiados comunicarseweb.com http://comunicarseweb.com.ar/?Indra+desarrolla+una+soluci%F3n+de+seguridad+para+refugiados+&page=ampliada&id=816911/01/2012Indra crea un dispositivo de seguridad para proteger los campos de refugiados DIARIO MEDICO.com http://www.diariomedico.com/2012/01/18/area-profesional/entorno/indra-crea-dispositivo-seguridad-proteger-campos-refugiados18/01/2012Indra lidera un proyecto para campos de refugiados diariovasco.com http://www.diariovasco.com/v/20120112/economia/indra-lidera-proyecto-para-20120112.html12/01/2012Indra lidera un proyecto para campos de refugiados elcomercio.es http://www.elcomercio.es/v/20120112/economia/indra-lidera-proyecto-para-20120112.html12/01/2012Indra lidera un proyecto para campos de refugiados Diario montañes ----------- 12/01/2012Indra desarrolla una solución de seguridad destinada a campos de refugiados Gigatronic ----------- 11/01/2012Campos de refugiados más seguros La Razón ----------- 04/03/2012Further details see deliverable D8.1 in Appendix 3. Project posterDuring the project several posters have been presented:POSTER TITLE EVENT/LOCATION PARTNERSINVOLVED DATE“European School of Antennas. Frequency Domain Techniques for Antenna Analysis” EuCAP 2012 (6th European Conference on Antennas and Propagation) / Prague (Czech Republic) UFL 26-30 March 2012“Seguridad total para campos de refugiados y entornos impredecibles” HOMSEC / Madrid (Spain) NTGS 12-15 March 2013“Basylis. Protection of critical infrastructure using rapid deployable arrays of sensors” Microflown Office (Internal promotion) / Netherlands AVISA Life time of project“Mobile, autonomous and affordable system to increase security in large unpredictable environments” Counter Terror / London (UK) INDRA 24-25 April 2013Further details see deliverable D8.1 in Appendix 4.Scientific papersDuring the project scientific papers have been published:[1] University of Florence, Innovative Double Spiral LP-RLSA Design for Low Cost Transportable Radar, Antenna Applications corner of IEEE Antenna Propagation Magazine.Further details see deliverable D8.1 in Appendix 5.Conferences and project presentationCONFERENCE PRESENTATION OF BASYLIS DATE PLACE PARTNERS INVOLVEDEuCAP 2012 – 6th European Conference on Antennas and Propagation NO 26-30 March 2012 Prague, Czech Republic UFLEuCAP 2013 – 7th European Conference on Antennas and Propagation YES 8-11 April 2013 Gothenburg, Sweden UFLThe 2012 International Crime Science Conference YES 4th July 2012 British Library, London UCLFinnish Border Guard YES 8th March 2013 Mirasys HQ, Helsinki MirasysHOMSEC NO 12-15 March 2013 Madrid, Spain NTGSWorkshop YES 16th April 2013 Madrid, Spain ALLCounter Terror YES 24-25 April 2013 London, UK INDRA, TERMA, MIRASYS, UPMFurther details such as information of Conuter Terror Expo see deliverable D8.1.Further details such as presentation see deliverable D8.1 in Appendix 6.LeafletsDuring the project two leaflets have been created. The first leaflet created has been used during the most part of the period of the project.The second leaflet has been created in order to be used during the Counter Terror Expo 2013.3D AnimationAt the final part of the project an animation 3D has been created by Indra. In the next photographic sequence, the most relevant images of 3D animation are shown in section 1.6 (subsection 1.6.3).Exploitation of results The results obtained in the project follows: The sensors developed for perimeter security, covering from (3 kilometers to 200 meters): Radar: Object detection system that uses electromagnetic waves to identify the range, direction and speed of moving objects. The radar will be able to operate both on civil and governmental reserved frequencies changing the transceiver module. Detection system that uses laser waves to identify the range, direction and speed of moving objects. The problem of ladar without elevation control is the width of the laser ray. It is so reduced that any obstacle or slope in the terrain blinds it, due to the line-of-sight coverage. The new development will implement a vertical angle control system for the laser beam to be established at the most convenient height (over/below 0), increasing the adaptability of the ladar to the terrain. Camera allows visualize the environment in order to have a visual control and analyze the different situations.The sensors focused for inner areas protection are (covering from 1 meter to 50 meters): Acoustic Vector Sensor (UNAVE): Acoustic vector sensors can be used to locate, identify and track in a 3 D space a certain number of sound sources simultaneously. The sensor will be in UGS format, incorporating communication and batteries in the same enclosure. COTs Integration Board: This board allows the easy integration of alarms from COTs sensors in the system, as microwave barriers, access control or volumetric. The dry-contact alarm output of the COTs sensor is connected to the COTs Integration Board which incorporates the communication interface to transmit the alarms to the Control Centre. Metal UGS Weapon Detection System: Metal objects detector based on UGS format, incorporating communication and batteries in the same enclosure. It can be camouflaged under the ground, detecting if the people in its range are carrying metallic objects. Seismic UGS. The Seismic UGS System is a low cost self-powered system with integrated communications easy to deploy and adaptable to the terrain orography. It can be used for perimeter protection using buried seismic sensors, hidden underground, for critical infrastructure protection as buildings, military camps, power stations, and nuclear power stations. Besides, seismic UGS sensors have been designed for sustainable operation in adverse conditions: rain, snow, thunderstorms, flooded terrain etc.The sensors developed for Personal Security are: Portable Units: Portable bracelets of fixed autonomous system, incorporating battery and wireless communications interface. They will have a button to generate an alarm on risky situations. Also sabotage and low battery alarms will be incorporated. The Control Centre will get the alarm and an estimation of the position based on radio location using the communication network, or the integrated GPS position if available. A continuous position data can be sent for special risky applications, for example protection of UN staff. Main Wireless Network: A wide range communication network will be developed to can communicate all the sensors alarms to the Control Centre, including the Portable Units. This network will also incorporate the radiolocation functionality to avoid the use of GPS in bracelets. The resolution of the system for positioning will depends on the range of each communication antenna, but the system will allows to overlap antenna coverage’s to increment the resolution of the system in specific locations (latrines…) using directive antennas.The software module of BASYLIS consists of: Image automatic Alarm classification: The system will automatically verify the alarms and classify the detected objects as persons or vehicles using image processing. A long range camera will be automatically positioned with the alarm to be classified. Simulation Engine: The simulation engine for all the alarm scenario and sensors simulation will allow the testing of the software for multiple scenarios and the faster developments of the BASYLIS project, enabling the start of the software development before the sensor prototypes are finished. Multitracker/Alarm fuser: This software module will integrates all the sensors alarms, including the COTs ones thought the COTs Integration Board. The software module will also incorporate tracking functionality, comparing new alarms with the old one to identify if they come from the same object. The output of the system will be a sensor unified alarm message (virtual sensor) and the object (not the alarm) which has generates it. Behavioral Analysis: The behavioral analysis automates the identification and classification of suspicious behavior of objects from the multitracker/alarm fuser. The behavior analysis will allow an early identification of potential threatening situations and the behavioral analysis of persons and groups of people, to detect demonstrations of attacks. Human Machine Interface (HMI): The user friendly HMI will incorporates a point & click functionality over a geo-referred map for sensor positioning and configuration in the system, decreasing the deploy timing. The HMI will show an image from the camera for manual alarm verification, and graphical configuration of alarm areas.Exploitation strategyThe exploitation strategy defined for promoting the results obtained in the project follows: Attendance to HOMSEC. HOMSEC is the most important event in Spain for Security and Defence technologies. NTGS presented in a stand a poster of the ladar developed in Basylis. Due to the small size and easy deployment of the system, it was shown as ideal solution for protection/surveillance of temporary camps of any nature, due to disasters, military deployments, makeshift camps, etc. Workshop with end users: a workshop with end users has been held the 16th of April 2013 where BASYLIS security solution has been shown by means of a real demonstration of the system. Several potential stakeholders were invited. Counter Terror Expo: BASYLIS project has participated in the Counter Terror Expo, the 24&25 of April 2013 in London. The Counter Terror Expo is the largest international event to mitigate threats, protect against terrorist attacks and understand current modern day risks in a secure environment. Indra, Terma, Mirasys and UPM have participated in the event. Approaching this event has been very relevant for promoting the project, as it is one the biggest trade fares in the world on security and defense oriented technologies. Thanks to the project stand and the 90 minutes slot for project presentation, BASYLIS project reached to a very wide audience. UCL is planning to write two publications based on the activities from WP6. The first will be a journal paper which will consider the development of the following behavior model and the development of the classifier to detect it. The evidence from the first study showed that the standard models for following behavior do not describe how people genuinely follow one another and our preliminary experiments suggest that high-performance classifiers can be developed. We will also investigate the feasibility of writing a paper on the multitracker. The formulation of entities, tracks and sensor reports is a generalization of tracklet-based and track segment graph-based approaches to fusion, and also tests the posegraphs in a more dynamic context than they are usually applied in. In the frame of the master “Communications Technologies and Systems” at the Technical University of Madrid –the largest Spanish University for engineering-, BASYLIS has been selected as a case study of integration of existing technologies for improving system capabilities. This activity will be developed in the second semester of the academic year 2013-2014 and it will be followed by PhD students and engineers from companies related to technology developments and security-system developments.This short course will consist of a presentation of the problem solved in BASYLIS –an affordable solution for providing security in unpredictable environments-, the different alternatives, and the solution followed by BASYLIS consortium.List of Websites:A public project website was developed by the project coordinator INDRA. The web url is: http://www.basylis.european-project.eu/homepage.phpContentsWebsite sections created are the following: Homepage: in this section is possible to read a brief summary about the Basylils project, the most important description, the technologies used, the objectives... Partners: in this section it possible find the partners involved in the project and information about them. Project Overview: this section is composed by two part:o Developed in Basylis: the technologies employed in the project are described in this section. Each sensor or development is focused to a determinate mission: Perimeter Security, Inner Area Protection, Personal Security and Software and Image Processing. A brief description of each system is carried out in this sectiono BASYLIS system: the whole system allows to have a global vision of the project. The connexion between modules, the relationship between them and the schedule is shown in this point. Dissemination: this section allows showing the events carried out for the project presentation in the society. ContactFurther details see deliverable D8.1 in Appendix 2. Documents connexes final1-d9-5-finalreport-eu-figures-v2.pdf
