Final Report Summary - INDECT (Intelligent Information System Supporting Observation, Searching and Detection for Security of Citizens in Urban Environment)
Executive Summary:
The EU-funded project “Intelligent information system supporting observation, searching and detection for security of citizens in urban environment” (INDECT, http://www.indect-project.eu/) worked on developing solutions designed for police and other law enforcement authorities in the EU. Focusing on automatic threat detection in urban environments, the project team developed a set of tools supporting decision-making in counteracting threats and criminal activities.
European scientists and researchers developed solutions and tools for automatic threat detection. The primary objective was to develop advanced and innovative algorithms for human decision support in combating terrorism and other criminal activities, such as human trafficking, child pornography, detection of dangerous situations (e.g. robberies) and the use of dangerous objects (e.g. knives or guns) in public spaces. Efficient tools for dealing with such situations are crucial to ensuring the safety of citizens.
A significant part of the project was dedicated to the development of tools and methods for data and privacy protection. The processed information is protected before its transmission or storage to prevent any attempts at unauthorized access. Dedicated tools are being developed to protect citizens’ privacy in areas covered by visual monitoring systems. The main tasks of INDECT can be divided into 3 categories as shown below:
The first category, “Intelligent Monitoring for Threat Detection”, is dedicated to the automatic detection of real world threats (video and audio analytics).
In the second category of “Threat Detection in Computer Networks” (Internet Forensics) the research focused on identification of sources, which distribute child pornography, or information related to the illegal trade in human organs. Project members developed software in order to combat organized crime. One of the main tools developed within this category is INACT, a system for rapid detection of child pornography on storage devices (mainly hard drives) captured by police forces on the basis of a warrant. INACT is especially useful if the illegal content is hidden in large data volumes.
The third category was focused on the development of “Data and Privacy Protection” techniques. Project members elaborated on data and privacy protection techniques using digital watermarking and cryptographic algorithms.
The main tangible INDECT results are over 50 working prototypes. Some of them were presented and awarded at international fairs and conferences, including:
Gold Medal at Brussels Innova 2010,
Gold Medal at Poznan International Fair 2010,
Gold Medal at Technicon 2010,
Gold Police Star at Europoltech 2011,
Gold Police Star at Europoltech 2013,
Silver Police Star at Europoltech 2013.
Gold Medal at International Concours Lépine 2014”, Paris
Award of Polish Ministry of Science and Higher Education)
Two gold medals at BRUSSELS INNOVA 2013
The Gold Police Star and The Silver Police Star Award at Europoltech 2013, Warsaw, ”SUPERMODERN 2013” Programme
INDECT research teams published over 400 scientific papers - also in such prestigious journals as “IEEE Transactions on Information Forensics” and “Security and IEEE Transactions on Image Processing”.
INDECT organised annual international MCSS conferences in 2010, 2011, 2012, 2013 and 2014.
Other INDECT activities related to dissemination activities comprised:
Meetings with Members of European Parliament
Meetings with Members of National Parliaments
Participation to events related to privacy and data protection such as “Computers, Privacy & Data Protection” Conferences held in Brussels yearly and yearly working meeting with Polish Members of the European Parliament organised by Data Protection Officer.
INDECT team actively cooperated with bodies responsible for personal data protection, i.e: Polish Inspector General for Personal Data Protection, Article 29 Working Party and NGOs (such as Panopytkon).
Project Context and Objectives:
The Internet and globalisation have brought the world closer in many ways, but they have also empowered international crime rings and terrorist groups. Sophisticated theft, child pornography, terrorist attacks and human trafficking have taken on an alarmingly global dimension, prompting a need for high-tech solutions to neutralise such threats.
The EU-funded project “Intelligent information system supporting observation, searching and detection for security of citizens in urban environment” (INDECT, http://www.indect-project.eu/) worked on developing solutions designed for police and other law enforcement authorities in the EU. Focusing on automatic threat detection in urban environments, the project team developed complex software to support decision-making in combating criminal activities and thwarting terrorist attacks.
INDECT project involved European scientists and researchers in the development of solutions to and tools for automatic threat detection. The primary objective was to develop advanced and innovative algorithms for human decision support in combating terrorism and other criminal activities, such as human trafficking, child pornography, detection of dangerous situations (e.g. robberies) and the use of dangerous objects (e.g. knives or guns) in public spaces. Efficient tools for dealing with such situations are crucial to ensuring the safety of citizens. A significant part of the project was dedicated to the development of tools and methods for data and privacy protection. The processed information is protected before its transmission or storage to prevent any attempts at unauthorized access. Dedicated tools are being developed to protect citizens’ privacy in areas covered by visual monitoring systems.
INDECT - Main Research Areas
The main tasks of INDECT can be divided into 3 categories as shown below:
The first category, “Intelligent Monitoring for Threat Detection”, is dedicated to the automatic detection of real world threats (video analytics). Contemporary Closed-Circuit TeleVision (CCTV) monitoring systems are mainly based on the continuous observation approach, where the monitored area is under constant supervision by the operator. If the operator is dishonest, this can potentially lead to privacy abuse. INDECT improved CCTV strategies to focus on potential threats rather than on continuous monitoring of day-to-day activity. The system automatically alerts operators in case a threat is detected, enabling them to make the final decision on whether to react in case of a real threat. The INDECT project adopts a new approach based on threat monitoring (also known as “black/dark screen monitoring”). In contrast to the traditional approach, it focuses on potentially threatening situations, and gives the operator access to the video only when his attention is actually needed. The system automatically detects dangerous events and alerts the operator, who makes the final decision about the action that needs to be taken. The process is carried out in accordance with all relevant regulations. The research conducted within the project aimed to improve the efficiency and accuracy of existing video monitoring systems. INDECT was not installing any cameras in the EU; nor was it filming people at random. It was not connected to any existing database or social network.
In the second category of “Threat Detection in Computer Networks” (Internet Forensics) the research focused on identification of sources, which distribute forbidden content (such as child pornography), or information related to the illegal trade in human organs. Project members developed software in order to combat organized crime. One of the main tools developed within this category is INACT, a system for rapid detection of child pornography on storage devices (mainly hard drives) captured by police forces on the basis of a warrant. INACT is especially useful if the illegal content is hidden in large data volumes.
The third, final category was focuses on the development of “Data and Privacy Protection” techniques. Project members elaborated on data and privacy protection techniques using digital watermarking and cryptographic software. This would help combat malicious tampering with multimedia, improving security of data transmission and storage. The system was designed to protect against the exploitation of sensitive information such as license plate numbers or faces. There are two main research directions related to this category – development of digital watermarking, and cryptographic algorithms. The digital watermarks are used to secure multimedia content against malicious tampering, and to provide a novel privacy protection mechanism. The approach adopted enables sensitive information, such as faces or license number plates, to be collected and displayed to authorized system users only. The development of new cryptographic algorithms delivers enhanced security for both data transmission and storage. A modern symmetric cipher has been designed, verified, and implemented both in software and hardware. This research also addresses the potential of adopting quantum cryptography to further improve the security of the solutions developed.
Overall, the project developed its strategies and tools with data protection and privacy in mind. Measures adopted included minimising unnecessary video recording and protecting information before it is transmitted or stored. The project’s results were highly received and demonstrated great potential in achieving their aims. The proposed technologies won several prizes between 2010 and 2014, gaining interest of end users and relevant stakeholders.
INDECT could prove particularly useful in alerting authorities about incidents on train platforms or where hooligans throw objects in stadiums, for example. This can be achieved by enhancing existing systems without adding extra surveillance cameras.
Varied benefits range from detecting abandoned luggage to analysing threatening sounds. Once exploited the solutions will support the control of cybercrime, terrorism and hooliganism.
Main Project Objectives and Results
To develop an intelligent information system for automatic detection of threats and recognition of criminal behavior or violence.
New, award-winning, methods have been developed, for tracking of objects of interest (source of the sound, or a “trigger” of visual event) by Pan-Tilt-Zoom cameras pointing automatically at given location. The location is provided by sound source detection and localization methods, or by video analysis of the scene observed by a fixed megapixel camera. Video processing has been supported by new solutions for vital problems of video quality in distributed monitoring: automatic assessment of streaming conditions and video compression artifacts by means of Quality of Experience methodology. Algorithms based on the all above-mentioned methods, together with secure mechanisms for acquisition, storage and effective sharing on demand of multimedia data gathered, have been implemented in 15 Node Station prototypes that have been developed, installed in cooperation with Partners, preliminarily tested in simulated real conditions, and prepared for extensive tests phase. Finally, the first prototype of a Central Station has been developed, serving as a management point in the distributed pilot installation of the monitoring system.
Overall self-organising computer network architecture model has been proposed. A biometric features analysis component, based on video and image information, has been proposed. Similarly, a multimodal features analysis component has been provided. Finally, a prototype of automatic event detection system has been developed.
Furthermore, proposed logical structure of the Unmanned Aerial Vehicle (UAV) systems has been elaborated. Systems architecture has been described. Proposed algorithms for positioning, tracking and predicting of the position of tracked objects have been reported. Similarly, the proposed algorithm and mechanisms for wireless transfer of partitioned spatial data have been provided as well. Also, the proposed algorithms and methods for autonomous steering and navigation of UAVs have been reported. Moreover, the proposed algorithms for mission planning for groups of autonomous UAVs have been delivered. Finally, the prototypes for the implemented subsystems have been elaborated.
To develop new methods and techniques providing tools to support activities of police officers, including tools for threat detection in the Internet
INDECT developed high precision crawler technologies navigating the Internet World Wide Web (WWW) websites, by following the hyperlinks and analyzing the content found in the websites, in order to find highly specific information on the Internet, allowing for mining for and detecting suspicious websites. These processes can be based on textual or multimedia searching.
As far as textual searching is concerned, software for learning relationships between people and organizations through websites and social networks has been developed. The software is able to learn behavioral profiles of known criminals.
As far as multimedia searching is concerned, software has been developed, allowing for integration of developed different multimedia analysis tools (called plugins). Different plugins have been integrated or are foreseen: image/graphic recognition, face detection/recognition, symbol detection, and automated child pornography detection.
In order to allow continuing investigations related to child pornography, INDECT developed Advanced Image Cataloguing Tool (INACT). The purpose of the tool is to catalogue the images found at the arrested suspect’s HDD and to compare the images against the images gathered during the investigation. The outcome of such search can be used to prove that the suspect is in the possession of the forbidden content and/or has been actively sharing the forbidden content.
For targeted criminal analysis, INDECT developed new components for the award-winning LINK analytical platform. The components allow for registration of the user operation history, executing sequences of operations on large volumes of data, and integrating selected Internet data sources (e.g. register of companies). The criminal analysis platform has been integrated with the pattern recognition application, communicating with a police officer via multi-touch-enabled analytical desktop interface. Further targeted criminal analysis applications have been developed, including: an extended version of the KASS Social Network Analysis system with the functionality dedicated to the analysis of data coming from Internet blogs, abcLog application for retrieving information about IP addresses for large volumes of data based on Internet data services as well as a system supporting management and strategic analysis of data related to criminal group activities.
Moreover, a prototype of the platform for the capture and analysis of malware has been created.
Finally, INDECT developed tools to process the information provided by citizens via emergency website.
To develop techniques for data and privacy protection in storage and transmission of data.
INDECT developed content access protection system, being a new concept of reversible image censorship, which uses a reconstruction reference embedded in the image by means of a digital watermark to restore the original appearance of the censored fragments. The principal application of this scheme is to protect privacy in video surveillance systems.
The proposed content protection scheme consists of two modules: an encoder and a decoder. The former is used just before the format conversion step which prepares the images for storage or transmission. The latter is a client-side module which, upon successful authentication of an authorized user, is capable of decoding the additional information from the watermark and performing content reconstruction.
INDECT has implemented a proof-of-concept prototype for protecting privacy in intelligent video surveillance systems. The system detects and tracks sensitive patterns (e.g. faces, car license numbers plates) and performs real-time anonymization.
Content access protection is one of the most interesting and most promising watermarking-related research topics carried out by the INDECT project. It addresses the privacy issues, which have become one of the most important aspects of the ongoing broad implementation of video surveillance systems.
To elaborate procedures and tools for making information more secure from unauthorized access and use.
All possible aspects of information security in INDECT project were defined. VPN types and technologies, analyzing their usability for INDECT project were specified.
Specification of simulator for cryptographic algorithms and a testing-tool, explaining the software to be used was done. Mathematical models as well as the GUI of components were provided.
Component for development and tests of a new Quantum cryptography protocol were delivered. A methodology, key parameters and simulation and measurement software tool for the existing and new protocols were elaborated.
Terminology of European ID authentication and identity management making an analysis of its components and giving the conceptions for Liberty Alliance, WS-Federation и SAML were defined. Subsequently recommendations for development and implementation were done as well as the relation between European ID authentication on one side and identity management and other WPs on the other.
Different aspects of Security and Authentication in Mobile Adhoc networks were specified. Blueprints of the secure multi-path routing protocol for ad-hoc networks were implemented.
New block cipher was constructed and evaluated with respect of its vulnerability to errors. Innovative encryption/decryption algorithm was implemented. Also tests and simulations have been performed to check and evaluate the security levels. The simulation software was also elaborated for evaluation and comparison of different cryptographic algorithms.
INDECT successfully passed the Final Review of the project, which was held in Krakow in June 2014 (3 reviewers from European countries took part in a panel meeting) and Ethics Reviews: including the one held in Brussels in March 2011 (9 reviewers from European Commission and from several European countries took part in a panel meeting) and the final Ethics Review.
Project Results:
This section provides a short overview of selected INDECT project research areas, results and brief description of prototypes.
Node Station Framework for Multimedia
GUT Multimedia System Department team developed an Intelligent Surveillance System for automatic event detection. Real-time and concurrent processing of audio fulfils the goal and video streams, for object detection and tracking, by geo-positioning of detected objects for automatic PTZ camera positioning, and adding Bluetooth object identification. The software was developed and tested in a dedicated Node Station (NS) framework (hardware and software solution). In the period of 2013-2014 the NS was tested in real conditions, with a focus on algorithm optimization, and user accessibility, and was integrated with other elements of the INDECT Project.
The NS system was tested in outdoor and indoor conditions, by performing staged events in a controlled environments (access limited only to informed test participants). Correct handling of data streams from video IP cameras, thermal cameras, acoustic sensors, and Bluetooth transceivers were positively evaluated. The multimedia streams processing algorithms and the client application (video and audio overview and notifications) were developed as a multiplatform software (Linux, Windows, and smartphones are supported). Documented case studies include monitoring of buildings and their vicinity, parking lots, banking room environment. Experimental results were published in journals and presented on conferences and documented in INDECT reports (D1.5 D1.6 D1.7 D1.8 and D7.7 D7.8).
Real conditions evaluation of the Node Station framework was based on two setups, Gdansk Science and Technology Park, and campus of Gdansk University of Technology. The final result is a fully functional system of Node Stations, allowing for distributed processing of multiple audio and video streams, and transmission of detection results. To disseminate the solution GUT and AGH organized a Node Station programming workshop in Krakow (section 3.4.3).
Contacts and cooperation with End-Users, including the meeting at GUT and live presentations of NS framework, resulted in a report D1.7. „Complete documentation and manuals for all classes of users of the system”. Cooperation between Partners of WP1 and WP7 allowed to define requirements reported in D1.5. „Specification of procedures for data exchange including definition of system usage with respect to the law and police regulations”. Node Station framework developed in WP1 was integrated with the result of WP7, namely the Integration Platform, aimed at notifications presentation (documented in D7.7 “Prototype of complex multimodal features detection system”).
The Passive Acoustic Radar application was extended with PTZ moving camera, and advanced video processing algorithms, resulting in a “System for acoustic and visual remote observation”, that was exhibited at Europoltech fair, and awarded Golden Police Star.
The communication with other modules in the NS Framework was finally defined. The timestamp holding mechanism during frame processing in the audio events detection module finalized. The decoder module was optimized for lower computing requirements. The front-end module was extended for parallel feature extraction, so different kind of features could be extracted in real-time and combined in resulted feature super vector. New parallel front-end module and demonstration GUI interface realized.
We analyzed the impact of the system bandwidth on the ToA and final node distance estimation accuracy and for NLoS mitigation problem, especially in case of NLoS communication with UDP, the low-complexity NLoS mitigation technique for ToA estimation has been proposed. Performance analysis of the Anchor-free Localization Algorithm with Low-Complexity Method for node Distance estimation enhancement using ToA has been done. Testing of the designed localization method by the UWB802.15.4a model for wireless channel impulse response generation in the OFFICE, RESIDENTIAL and INDUSTRIAL environment. Modification of the reactive MANET routing protocol named dynamic source routing protocol (DSR). We integrate possibilities to use opportunistic routing of the messages in the case if the routing protocol cannot find communication paths. Implementation of the designed selection the best candidates nodes methods and testing of the proposed algorithm in the OPNET modeler simulation environment. Our research activities were focused on the publishing of the collected results about localization of the mobile terminals and temporary disconnected mobile ad-hoc networks (MANET) in research conferences.
Identification and Observation of Mobile Objects in Urban Environment
This part of the research concerned the Monitoring of Physical Objects and Detection of Threats and mainly related to detection, recognition and prevention methods deployed in urban areas.
The initial research activities were mainly concentrated on creation of self-configurable ad-hoc sensor network architecture, which will be able to detect many features of objects simultaneously. The issue of limited capabilities in autonomous target searching for targets with defined characteristic has been solved by applying modern methods and algorithms for image processing for automatic locating and tracking of targets.
Work in all areas has been continued focusing on network centric system, especially on multipurpose city sensor. We have been working on an algorithm for redetection a visually tracked object after a period when it is occluded. We have been also testing and analysing communication algorithms in MANET, based on Time of Flight estimation, namely MUSIC, ESPRIT and Matrix Pencil. We have been also performing test flights of Indect UAV in order to gather image data and to test available stabilization and tracking algorithms.
The following tasks were dedicated to the construction of prototypes of sensor network built from - master sensor station (communication backbone), observation sensor station (plate recognition) and mini tracking sensors. We have been also working on different techniques for LoS/NLoS (Line of Sight/NoN Line of Sight) channel identification and the ToA (Time of Arrival) node distance estimation.
In the next phase research concerned the possibilities of the Time of Arrival estimation from the channel impulse response (CIR) of the narrowband systems. The research has been also focused on the issues of multipath routing in opportunistic networks. Emphasis was also placed on the security and trust models for opportunistic networks. The results were published in the Journal of Electrical Engineering and in Oradea. Furthermore extensive research on the recognition system has been conducted. This was completely new chapter of research in this field. We have also created series of schematics, revisions, devices and casings for sensor network.
Other research activities were done in the following areas:
node distance determination enhancement by using Time of Arrival estimation in the case of Non Line-of-Sight communication with Undetected Direct Path (NLoS-UDP),
preliminary tests of sensors - filters that evolved with use of the Genetic Programming used in video-sensor station.
Design and development of a tool for monitoring Wi-Fi hotspots to detect unauthorized access and unusual or suspect behaviour
We have designed a method to authenticate Open Wireless Networks to clients. The proposed solution is a defence against any kind of Twin Evil attacks on an Open WLAN. The idea is that clients can trust the network they want to associate with without the fear of being spied or infected directly by an attacker who makes a Twin Evil attack. The authentication of the Open WLAN is based on certificates delivered by a Certificate Authority. The solution permits the authentication of the wireless networks, which avoids association with a fake AP. It also permits to encrypt the communication between a client and the associated AP to prevent passive eavesdropping. We have implemented the proposed method on Linux. To implement the solution on the AP, we use a computer with Hostapd and Compat-drivers on the computer of the client.
The participation of UC3M in WP3 is mainly related to the development of the INDECT Lawful Interception Platform (ILIP).
ILIP is a novel platform for the Lawful Interception of data communications that will enable European Police forces to monitor in a cost-effective way the Internet activities of criminals, which increasingly employ these communication networks to, both, coordinate crimes in the real world and even commit crimes directly over the Internet.
Two types of elements compose the ILIP platform:
ILIP Capture Stations. These probes are placed at the Point of Presence (PoP) of an Internet Service Provider (ISP) and capture all the traffic from the clients of the ISP covered by a valid DWW. Seized data traffic is digitally signed, time-stamped, compressed, and encrypted before being sent to the decoding server assigned to the case.
ILIP Decoding Server. Each investigation case has an associated Decoding Server that, from the packets captured by the monitoring stations, reconstructs the communication flows, and decodes the contents being exchanged by the suspects under investigation. The decoded contents will be indexed and permanently stored in a database to ease its access by authorized Police Analysts.
The development of the ILIP platform has been completed in 2013, covering both elements of the ILIP platform. Moreover a demonstration video that shows the main features of ILIP and the developed prototypes has been recorded during the first months of 2014, in order to be shown during the Final Audit to be held on Krakow on the 13th of June.
Details about the ILIP user interface and the protocol between the ILIP decoding server and the remote pre-classification plugins are described in deliverable D3.9 “Summary report on interfaces, components and implemented algorithms” that UC3M has co-authored.
We have designed and tested a new password attack for the chosen language environment. The algorithm uses newly built character frequency tables that allow testing the strength of passwords derived from the chosen language. This approach assumes that passwords are constructed as meaningful combinations of letters. This is very important, since for example testing WPA-PSK passwords is very computationally intensive so efficiency is necessary.
The new statistical approach used for attacking passwords was successfully tested for desired language and also for desired language group (for example a language model of Slovak, Czech and English words) and best practice to create very effective password candidate lists for specified language group proposed in the journal paper.
Next, the new statistical approach used for attacking passwords in specified language was successfully evaluated and a best practice to create a very effective password candidate lists for specified language presented on IEEE IWSSIP conference. The research in the area of statistical attack for specified language continued by analyzing textual databases and analysis of different language models.
New statistical approach used for attacking passwords used in computer networks was successfully implemented and best practice to create very effective password candidate lists for specified language proposed in the paper.
A journal paper named “Security Audit of WLAN Networks using Statistical Models of Specified Language Group” published in Journal of Electrical and Electronics Engineering.
A paper named “Statistical Models Based Password Candidates Generation for Specified Language Used in Wireless LAN Security Audit” presented on 20th International Conference on Systems, Signals and Image Processing in Bucharest. The language models evaluation for different domains.
Social Threat Monitor (STM) is a Web system incorporating a relational database and GIS component for deregistration of user provided data on threats. In contrast to typical Crime Mapping Systems — where only authorized services, such as Police, are allowed and responsible for selective threat reporting — in case of STM all registered citizens can input threat data of interest. Registered data can be spatially visualized allowing easy browsing of reports. Wide, distributed and unlimited access makes this kind of service a new quality on the way to threat monitoring and safety improvement. However, due to huge volumes of complex, multi-aspect data, automated knowledge management appears indispensable. A paper outlining the ideas for a Rule-Based System for report processing and knowledge management has been submitted to MCSS 2013. A new taxonomy of rules for complex knowledge management tasks was put forward. The rules are used to assess report admissibility, access restriction, perform basic processing and threat inference. Special ECA-type rules provide immediate reaction to dangerous situations and run statistical analysis of gathered data. The rules are written in Prolog.
The research started to investigate possible extension of the STM front-end towards mobile, context aware applications have been continued. With a rapid evolution of mobile devices, the idea of context awareness has gained a remarkable popularity in recent years. Modern smartphones and tablets are equipped with a variety of sensors including accelerometers, gyroscopes, pressure gauges, light and GPS sensors. Additionally, the devices become computationally powerful which allows real-time processing of data gathered by their sensors. Universal access to the Internet via Wi-Fi hot spots and GSM network makes mobile devices perfect platforms for ubiquitous computing. Although there exist numerous frameworks for context-aware systems, they are usually dedicated to static, centralized, client-server architectures. There is still space for research in a field of context modelling and reasoning for mobile devices. A novel, lightweight context-aware framework for mobile devices that uses data gathered by mobile device sensors and perform on-line reasoning about possible threats, based on the information provided by the Social Threat Monitor system developed in the INDECT project has been put forward. These concepts were put in a draft version of a paper for the MCSS 2013.
The server dedicated to hosting the Social Threat Monitor (STM) undergoes the usual maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The prototype of the INDECT system is now stored on the vigil server [vigil.ia.agh.edu.pl]. The hardware configuration of the server was given in the former report and has not changed. An improved access control policy and its implementation was reported in the last report. It was implemented on vigil, using the HTTP server. It limits the access of the service to dedicated INDECT users only.
Current maintenance of the vigil has been continued. It included periodical software upgrades, log analysis and rotation. The server dedicated to hosting the Social Threat Monitor (STM) undergoes the standard periodic maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The system runs on the dedicated Debian squeeze server, called vigil1 that provides the set of necessary tools that are relevant from the system point of view.
At least the following set of security measures need to be in place for any software described or produced within WP4: (1) the use of the software in a secure environment, (2) secure management of data sources by encrypting the used knowledge bases, (3) secure access policy, (4) secure storage of access logs and (5) anonymisation of Ids (email, URL, username etc) from data.
Combining confidence-rated user supplied knowledge from diverse sources and integrating
Work carried out within the University of York (UniYork), UK. In our method of text classification developed within Deliverable D4.12 the system takes as input the labelled data, extracts sub-sequences from the labelled data as features, and then learns a classifier using the features. The learned classifier is then used to predict the label for new unlabelled data.
The major functionality of the software is:
To provide user with a central interface to manage data of different types. For example, organ trafficking, illegal weapon trading, protests etc.
To provide user with an easy to use interface to collect data from different sources, e.g. from text file in a computer or crawl data from World Wide Web.
Use labelled data to train a classifier which can be further used to classify unlabelled data instances.
Allow user an easy mechanism to add, remove and modify the data.
Show suggestion if the classes of any of the labelled data needs to be changed. For example, if text content is labelled as organ-trafficking by the user and the software predicts that the content is normal and does not represent organ trafficking, then the software suggests such changes to the user.
Display data and output in an interface that supports search.
The system has been improved following extensive end-user consultations with PSNI. We have studied the user requirements carefully and realised that substantial effort had to be given to data collection mechanisms as opposed to data classification mechanisms. Thus, an active “human in the loop” system was devised resulting the FERRET data collection and classification engine. The prototype FERRET system is currently targeted for precise retrieval of data from social media sites such as Twitter but can be easily re-targeted for generic websites or other specific sites such as forums etc. The FERRET system takes into account the currently data gathering mechanisms and seamlessly integrates this within the data classification pipeline.
In the first stage, the system allows a case or profile to be constructed. Each case consists of a number of pattern identifiers e.g. “Kidney” and “Eyes”.
Each pattern id is defined in a separate “Patterns Section” shown below. This section permits both positive and negative keywords along with geo-location, diameter and sender id tags.
Once a list of patterns and entered, the system performs live search to retrieve matching tweets. The tweets can then be classified manually using the functionality provided by the system to manually tag these as show below.
The improvement in feature extraction is by augmenting sub-sequence feature with word-clusters set provided from CMU1 as shown below. By augmenting the cluster id in the feature dictionary now we can match two patterns even if they do not match lexically but they produce an identical cluster ID patterns.
Using sub-sequence as features creates a large feature dimension. Our experiment showed that the large feature dimension decreases accuracy. Thus we improve our earlier system by using a feature selection algorithm that identifies the features with minimum impact on classification and discards it.
We calculate F-score of each feature to measure the discriminating power of the feature. The larger the F-score is, the more likely the feature is more discriminative. We remove the feature with lowest weights and re-train the classifier. If the accuracy for the new classifier is increased, we keep the current feature set or else we retain the old feature set. The above process is repeated till there is increase in accuracy by removing features.
The accuracy of the system is significantly improved by employing feature selection and adding word clusters as features.
The software uses the trained model to generate keywords selected from the features with the highest weights during training. These keywords are provided as query to twitter. A search is performed a regular intervals and the results displayed in the table as shown below. Classification is also performed in the extracted Tweets to indicate whether the Tweet is positive or negative. The user can override these. These annotated examples are passed on to the earlier parts of the pipeline to continuously improve the classification accuracy.
Web System for citizen provided information, automatic knowledge extraction, knowledge management and GIS integration
The work described below was carried out at the AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, Department of Applied Computer Science – the GEIST Research Group: GEIST [http://geist.agh.edu.pl/].
The overall work concerned Social Threat Monitor (STM) – a GIS-based system for management of knowledge about threats provided by citizens (registered as system users). In the reporting period it was directed towards three main directions: knowledge management with rules, mobile context-aware applications and system maintenance.
Social Threat Monitor (STM) is a Web system incorporating a relational database and GIS component for deregistration of user provided data on threats. In contrast to typical Crime Mapping Systems — where only authorized services, such as Police, are allowed and responsible for selective threat reporting — in case of STM all registered citizens can input threat data of interest. Registered data can be spatially visualized allowing easy browsing of reports. Wide, distributed and unlimited access makes this kind of service a new quality on the way to threat monitoring and safety improvement. However, due to huge volumes of complex, multi-aspect data, automated knowledge management appears indispensable. A paper outlining the ideas for a Rule-Based System for report processing and knowledge management has been presented at MCSS 2013 conference. A new taxonomy of rules for complex knowledge management tasks was put forward. The rules are used to assess report admissibility, access restriction, perform basic processing and threat inference. Special ECA-type rules provide immediate reaction to dangerous situations and run statistical analysis of gathered data. The rules are written in Prolong.
The research started to investigate possible extension of the STM front-end towards mobile, context aware applications have been continued. With a rapid evolution of mobile devices, the idea of context awareness has gained a remarkable popularity in recent years. Modern smartphones and tablets are equipped with a variety of sensors including accelerometers, gyroscopes, pressure gauges, light and GPS sensors. Additionally, the devices become computationally powerful which allows real-time processing of data gathered by their sensors. Universal access to the Internet via Wi-Fi hot spots and GSM network makes mobile devices perfect platforms for ubiquitous computing. Although there exist numerous frameworks for context-aware systems, they are usually dedicated to static, centralized, client-server architectures. There is still space for research in a field of context modelling and reasoning for mobile devices. A novel, lightweight context-aware framework for mobile devices that uses data gathered by mobile device sensors and perform on-line reasoning about possible threats, based on the information provided by the Social Threat Monitor system developed in the INDECT project has been put forward. These concepts were put in a draft version of a paper for the MCSS 2013.
The server dedicated to hosting the Social Threat Monitor (STM) undergoes the usual maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The prototype of the INDECT system is now stored on the vigil server [vigil.ia.agh.edu.pl]. The hardware configuration of the server was given in the former report and has not changed. An improved access control policy and its implementation were reported in the last report. It was implemented on vigil, using the HTTP server. It limits the access of the service to dedicated INDECT users only.
Current maintenance of the vigil has been continued. It included periodical software upgrades, log analysis and rotation. The server dedicated to hosting the Social Threat Monitor (STM) undergoes the standard periodic maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The system runs on the dedicated Debian squeeze server, called vigil that provides the set of necessary tools that are relevant from the system point of view..
A new method for robust video watermarking was introduced. It uses a new developed Error Correcting Code and watermark embedding procedure in the frequency domain. The ingesting of watermark bits is done by QIM into the DCT spectral coefficients of low pass filtered stream. This low pass filtered video is generated by Inter-frame multi-level DWT.
The core idea of this scheme is the usage of the new developed Enhanced 3D Hadamard Error Correcting Code, for the purpose of video watermarking. Instead of ingest watermark bits it, Hadamard cubes are used as a code words. The code word length was selected to 512 bits, which implies the message length of 9 bits. The error correcting abilities of this Enhanced 3D Hadamard code are extremely good, especially in the case of burst errors. It is possible to correct more than 256 errors and even if the whole code word is corrupted it is possible to reconstruct it error free. This method can correct more errors as till now it was possible with standard Hadamard code.
The results are very promising. In an HDTV video sequence is possible to embed around 9112 bits (nearly 1,KB) watermark information per frame, which can be error free extracted after very strong compressions of the video. The compressed video has a data rate of 5Mbit/s (compression ratio 1:240). The embedded and compressed video streams show any visual degradation.
Search Engine for Documents Based on Watermarking Technology
The main purpose of this part of research was to elaborate a complete solution for searching, identifying and storing documents and multimedia contents to make the process of identifying criminals, hot objects, or just finding document, easier and faster. In addition, the high security for information flow needed by police and prosecutor offices will be offered.
The key objectives are indicated below:
design of new algorithms both for metadata storage in multimedia content (main aims: high encryption and redundancy whilst preserving perceived quality), which utilizes a watermarking-based approach with a background in current standards i.e. MPEG-7 and MPEG-21) and fast algorithms designed for searching such compressed and encrypted content with automatic authenticity checking;
developing high capacity watermarking technology using fast algorithms of selected transforms for the purpose of fast searching;
designing and implementing the method of software assistance support for secure searching of required persons and documents, including any security activities of police and prosecutors;
facilitating the use of existing assorted repositories designed for police and prosecutor offices by adding new interfaces and integrating currently separate systems;
establishing a coherent distributed system, this assists police and prosecutors in identifying criminals and hot objects in real time;
development of a semantic search engine for local and wide urban areas including Query by Example (QBE) techniques.
An agent technology is a suitable solution for large scale and complex commercial and industrial software applications. Agents in particular can support information retrieval in heterogeneous, distributed environments, where information is stored in different formats and types of relational databases, knowledge bases, etc. Retrieval of exact and suitable information requires analysing and rejecting redundant information or restoring of incomplete data. All these tasks can be fulfilled by multi-agent systems. Multi-agents systems are structured, co-operative organisations of different types of agents.
Quality of Experience research
The publication of QoE research – the website describing no-reference metrics was created (www.vq.kt.agh.edu.pl). A set of metrics were calculated, results were collected in a MATLAB file. The paper “Key indicators for monitoring audio-visual quality” was published. Metrics were implemented in the C++ language. Required data was prepared in MATLAB’s structures. Values of thresholds for two metrics were settled. The video library was created; it is available online at http://vq.kt.agh.edu.pl/videoLibrary.html. The first implementation of method for detecting logo was created. Implementations of method for detecting slicing and block loss artefacts were created. Values of thresholds for following metrics were settled: exposure time distortion, noise, slicing, freezing, and block loss. The paper: “Recent developments in visual quality monitoring by key performance indicators” was written. The video database at the PBZ server was cleaned up.
Some application and scripts for network measurements were implemented. Tests revealed high instability of packets sending and receiving time and due to this, the algorithm could not work. The research focused on other methods of available bandwidth estimation, which require streaming server modification and new features (like RTCP protocol) implementation. The changes or new implementation will also be required at client side. Another research results show that there is a possibility of using VTP (Video Transport Protocol) algorithm for video streaming. Analysis of packets streaming via Wi-Fi interface shown significant instability of packets sending and receiving times. The time differences were high enough to make impossible to implement the bandwidth estimation algorithms, which were based on fixed timestamps between packets. First tests of Extended RTP headers shown the possibility of using jRTPlib for this purpose and receive stream via standard video player, like VLC. A LibPacketAnalysis library was implemented which can be linked to any software and show the estimated bandwidth results. The code is in a development stage and has to be cleaned and documented. Using tc on output interface can be also a method for adaptive video streaming. When the player is MPlayer, there is a possibility to limit the bandwidth for particular stream instead of changing coding parameters. MPlayer displays the received whole frames like there was frame rate change.
As the project’s objective was to develop statistical models that will be able to estimate the quality of a video with regards to its usefulness in discerning visual information, for this purpose, some data must be gathered, which includes results of automatic quality assessment and recognition rates from experiments involving humans. This part of the project concentrates on performing calculations with objective measures and writing the results to a database.
A massive database has been created, storing several parameters coming from two different experiments: recognition of license plates, vehicle maker and vehicle colour in a parking lot, and recognition of different objects in different scenarios. Both experiments have been converted into a common format, as far as possible. In total, there are:
126 SRCs -> original video sequences
40 HRCs -> sequences modified (cropping, compression...)
1860 PVS -> derived clips
193 subjects answered in all experiments
69236 answers (data: rows in the database
This database is available in order to get parameters from there (such as bitrate, resolution, ...) and also to update it with other parameters (which need to be calculated).
Apart from that, all this data has been converted into a single numerical matrix in MATLAB, which will be the base for starting the modelling process.
Quality evaluation must focus on the areas where essential information is found. Some work has been done on following selected objects. It can be achieved in quite a simple way when standard foreground determination is possible. However, the method should also enable tracking in case of unstable background, and the objects may be nested in larger ones. For this reason, another approach tried was the use of optical flow. The implementation of this method found in MATLAB did not ensure sufficient accuracy. A more sophisticated algorithm is needed to reliably accomplish the task. Probably the best way to overcome the problem is to use a function from the OpenCV library.
There was also some attention paid to additional tools, like Kalman filter, which will be used to improve the accuracy and correct random errors. Methods of keeping track of multiple interacting objects were also analysed in case they are needed in the future.
To connect MATLAB code to database, external Java driver had to be installed and some configuration done. Available data about target recognition results is heterogeneous. Its structure is not certain at the moment, so the code for accessing it could not be completed. However, some of the design questions have already been resolved.
All the data available since the realization of the experiments (single values), i.e. answers from the viewers for every PVS (SRC-HRC), bitrate, resolution, scenario, etc. has been joined in a MATLAB structure to the results of having run the NR and FR metrics (single value for every frame -> arrays for every PVS). Three different sub-experiments (not in laboratory, at AGH University and by not practitioners respectively) from the objects recognition subjective experiment have been added to the two original sub-experiment (both in laboratory) included in the data. Taking into account the parking lot experiment, there are a total of 6 different experiments. Thereby, in just one MATLAB variable all the data required for the modelling process is accessible. This structure has a total of 69236 sub-structures with 23 fields each one: SN, SRC, HRC, recognized object, original object, bitrate, colour, make, file size, Levenshtein distance, viewer, resolution, scenario, correct, experiment, incomplete, blocking, blur, exposure, spatial activity, temporal activity, SSIM and VIF.
Analysing the likelihood of correct recognition from the 6 different experiments, the first decision taken has been to start searching for a model just for the 3 experiments, which have similar probability of correct recognition (around 80%).
First calculations (means, medians, and quantiles for NR and FR results) and conversions (creating arrays storing the answers for every PVS instead of doing it for every viewer) have been done to make easier to find the first model. Afterwards, just the mean will be used.
The first results showed us that there is no a clear correlation between any of the metrics used and the probability of correct recognition. This is the first conclusion exposed: better unique metrics should be used to obtain direct results.
Trying to create a generalized linear model was the next step. For this task, it was necessary to decide which set of videos was going to form the training set and which one the test set. Some linear models, starting from the most basic one, were tried. However, even the results from the most complicated of the models were not good at all. This model was taking into account every parameter, the square number of every parameter, and the total multiplication of all of them (both single and square).
As this model was totally unsuccessful, another approach was tried: neural networks. After running a program, which calculated, which neural network (set of parameters and number of hidden neurons used) was the one providing the best coefficient of determination R2, the best model found is the following.
Parameters used: Blur, Exposure, Spatial activity (SA) and Temporal activity (TA). Network structure, number of neurons in every layer: 7-8-1 (input layer – hidden layer – output layer respectively).
Algorithms dedicated to Classify Encrypted Traffic
In this work, we have proposed a payload-based method to identify application flows encrypted with the Secure Socket Layer/Transport Layer Security (SSL/TLS) protocol, which is a fundamental cryptographic protocol suite supporting secure communication over the Internet. Our approach consists of taking advantage of the information embedded in the SSL/TLS header to create statistical fingerprints of sessions to classify application traffic. We call a fingerprint any distinctive feature allowing identification of a given traffic class. In this work, a fingerprint corresponds to a first-order homogeneous Markov chain reflecting the dynamics of an SSL/TLS session. The Markov chain states model a sequence of SSL/TLS message types appearing in a single direction flow of a given application from a server to a client. We have studied the Markov chain fingerprints for twelve representative applications that make use of SSL/TLS: PayPal (an electronic service allowing online payments and money transfers), Twitter (an online social networking and microblogging service), Dropbox (a file hosting service), Gadu-Gadu (a popular Polish instant messenger), Mozilla (a part of Mozilla add-ons service responsible for verification of the software version), MBank and PKO (two popular European online banking services), Dziekanat (student online service), Poczta (student online mail service), Amazon S3 (a Simple Storage Service) and EC2 (an Elastic Compute Cloud), and Skype (a VoIP service). The resulting models exhibit a specific structure allowing to classify encrypted application flows by comparing its message sequences with fingerprints. They can also serve to reveal intrusions trying to exploit the SSL/TLS protocol by establishing abnormal communications with a server. Our analysis of the results reveals that obtaining application discrimination mainly comes from incorrect and diverse implementation practices, the misuse of the SSL/TLS protocol, various server configurations, and the application nature. Finally, even if we are able to identify some very reliable statistical fingerprints for selected applications, it is also possible to evade the classification by avoiding implementation mistakes and building the secure layer on limited, but widely used set of SSL/TLS states.
Reversible privacy protection with digital watermarking
The concept of reversible privacy protection is a new paradigm of selective content access protection. In traditional selective multimedia encryption techniques, the goal is to distort the whole image with the least possible computational effort. This is usually achieved by distorting just the visually significant portion of the data stream, i.e. the most significant bit-planes or low-frequency DCT coefficients. The proposed concept involves protecting access to selected media fragments only. This approach is motivated by the observation that in many cases the overall context of the presented content is still very informative and could potentially be useful for a more general audience.
The operation of the proposed reversible privacy protection uses a compressed reconstruction reference generated from and embedded in a digital image for the reconstruction of it’s original appearance. The detection and tracking module processes successive video frames and passes the locations of the privacy-sensitive regions to the privacy encoder using a binary protection mask. These ROIs are then blurred and the details necessary for prospective reconstruction are retained as a digital watermark in the remaining part of the image.The reconstruction is performed on demand on the client’s terminal only if valid access key is provided to the decoder.
The considered scheme operates on a discrete wavelet transform (DWT) image spectrum obtained using the biorthogonal CDF 9/7 wavelet. This results from the application of JPEG2000 for the compression of the stored frames.
In the first step encoder performs multi-level image decomposition. The coefficients that describe the sensitive regions and originate from all sub-bands from all decomposition levels are set to zero in the image spectrum. In the same time a certain portion of them is retained by means of a digital watermark. Generated watermark is then embedded using QIM algorithm in a way that each coefficient selected for information embedding carries a single bit of the watermark payload. In the last step the encoder synthesize image with watermark embedded in it.
The operation of the decoder starts with performing the same discrete wavelet transform as in the encoder. Taking advantage of the knowledge of the blurring procedure we can easily distinct protected and unprotected areas. Next, the decoder synchronize quantization step used in the embedding algorithm and extracts the watermark. In the last step, original content of the protected areas is recreated from the extracted watermark and the image is restored from obtained DWT spectrum.
For the proper encoder and decoder synchronisation and extraction only two parameters needs to be exchange between them i.e. protected image and the secret key. The rest of them is easily recoverable during extraction procedure.
Therefore the coefficient selection in the embedding step is determined in a pseudo random manner and depends on a secret key, it is required to posses a valid key to access protected parts of the image.
Visual description of this algorithm with some practical examples could be found at official INDECT YouTube channel: https://www.youtube.com/watch?v=IFEh8amSkpg
INDECT Prototypes
This section contains list of prototypes developed during the project. The prototypes were assigned to the following groups:
AUDIO EVENTS ANALYSIS
OBJECT DETECTION, RECOGNITION and TRACKING
CROWD BEHAVIOUR ANALYSIS
INDECT CRIMINAL ANALYSIS
SECURITY OF DISABLED PERSONS
IMAGE ANALYSIS FOR SECURITY PURPOSES
PROTECTION FROM DIGITAL THREATS
UNMANNED AERIAL VEHICLES (UAV)
INDECT INTEGRATION, DATA and PRIVACY PROTECTION
Group: AUDIO EVENTS ANALYSIS
Audio Events Detection, Classification and Localization
A module for real-time analysis of audio streams was presented. It performs detection, classification and localization of sound sources. It is trained to classify sounds related to threats (scream, explosion, gunshot, breaking glass). The information of localization is used to automatically position PTZ camera at the source and track it visually. The audio processing algorithms provide also a possibility of adjustable directional characteristic (overhearing a conversation from a user-defined direction in very noisy conditions).
Online Audio (Gunshot/Glass-breaking) Detection
Online audio detection demonstrated using graphical user interface of the detector running on one PC, and connected with audio cable with sound source, where different events are mixed with background noise with different SNR and sent to detector. The results contain also audio timestamp and event description.
Group: OBJECT DETECTION, RECOGNITION AND TRACKING
Visual Object Detection and Tracking - Event Detection and Automatic PTZ Camera Positioning
A module for real-time analysis of video streams was presented. It performs object detection, tracking, and classification. It is able to detect predefined events, such as: crossing a barrier in allowed/forbidden direction, various car parking events, abandoned luggage, person entering a road, and other. Detected event and object of interest are automatically tracked in a multicamera setup and available PTZ cameras can be automatically positioned to follow the object and provide high magnification image. Operation of the module was presented on live streams from outdoor scenes.
Surveillance and Localization of Detected Objects
The BUW node station is used for surveillance of an area of interest. Detected objects, e.g. pedestrians, will be located inside the world coordinate system, which can be expressed in GPS coordinates or in a kind of local world coordinate system using the metric “meters”. In other words, the local coordinate system includes a point in the field of view of the node station which has all coordinates equal to zero (center of the local coordinate system).
SSM station
SSM stations are designed as a highly modular system, that provides communication using different links. SSM are also equipped with sensors that can process signals received, examples include: plate recognition system or MiniDP devices. SSM stations are developed for obtaining data from tracking devices passing in its range. The station can be mounted at the entrances and exits of the underground structures, like underground garages, tunnels, subways or even at street crossings or junctions.
MiniDP tracking device
MiniDP device was developed to allow tracking of vehicles. To carry out the tracking function this device has been fitted with radio communication module working at ISM band, GPS receiver module, accelerometer and a microcontroller that integrates all of those elements and implements a communication protocol. All components were selected in terms of normalizing current consumption to provide the device with the longest battery life possible. By using the on-board accelerometer readouts microcontroller can determine whether monitored vehicle is not in motion and GPS module can be turned off in order to save power. One of the fundamental assumptions for the most effective design is to reduce the outline dimensions of the device to facilitate optimal placement in the tracked vehicle. With the use of neodymium magnets the process of placing MiniDP in a selected vehicle has been significantly simplified.
Vehicles Tracking Device
Tracking device was developed to allow tracking and positioning of vehicles and report through GPRS communication. The device is equipped with GPS positioning system, inertial positioning system, GPRS communication module, radio communication module working at ISM band, memory store and microcontroller. The microcontroller is responsible for the integration of the different subsystems integrated into the device and for establishment of the communications protocols.
Knife recognition
Recognition of objects resembling knives.
Mobile Object Tracking Module
The objective of Mobile Object Tracking module is to create a method which can track in a video sequence an arbitrary object pointed by a user. An implementation of this method will be used in Burzyk Unmanned Aerial Vehicle to follow any arbitrary target by the gimbal in such a way that the object will be in the camera’s field of view all the time. Therefore to cope with this task the real-time video processing algorithm running on the on-board embedded computer is the main requirement.
Group: CROWD BEHAVIOUR ANALYSIS
Crowd Monitoring
A module for crowd flow monitoring. It is based on processing of video signals from typical IP cameras (contrarily to dedicated Time of Flight cameras). The solution performs a real-time accurate detection and counting of people entering and exiting through a virtual gate.
Pedestrian Detection by Use of Boosting Algorithms Based on PTZ
Detection and tracking of pedestrians in the range of the PTZ node station, without limitation of moving or zooming the camera system (mobile camera). Detection algorithms are trained on examples of pedestrians and are not based on background subtraction. Combined with the localization feature, positions of detected pedestrians can be displayed inside a world coordinate system, e.g. a Google Maps image of an area.
People Detection and Tracking from a Top-View Position
This is a standalone system using Time-of-Flight (ToF) based camera images, captured from a Top-View position mounted perpendicular to the ground where people can reside. The camera measures distances (depth images) to each point in the real world. Filtering techniques are used to differ individual persons from a dense crowd of people in those depth images. Their trajectories are tracked over time and utilized for movement analysis, comprising counting of people, surveillance of hallways and detection of accumulations.
Embedded System for People Counting
The demonstrator presents implementation of the algorithm for detecting and counting people, who are passing under a camera positioned vertically down at the entrance to a monitored area (e.g. a building). The prepared algorithm takes the direction of the transition of a person into account. The model is realized as an standalone system i.e. on the Raspberry Pi microcomputer based on ARM embedded processor.
Crowd Motion Analysis
It presents intelligent video crowd movement analysis. The software collect information not only about the trajectory of movements of person, but also information about the frequency of selecting the routes, occupancy of particular areas of analyzed space and determining the areas and time periods. The system computes people density maps that allow to determine places where there is the greatest density of people or identify the occupancy of exits (including emergency exits) and escape routes without the need for laborious viewing of hours of video monitoring.
Density Measurements for Crowds based on PMD
Depth images are used to calculate the densities among crowds of people. With the projection of depths to a point cloud based 3D voxel grid, the occupancy is calculated and categorized to the resulting risk for persons.
Group: INDECT CRIMINAL ANALYSIS
The Lawful Wiretap Warrant System
A short video will illustrate the Lawful Wiretap Warrant system concept developed in INDECT and an example of hardware-supported content filtering and analysis.
System Babicka
System Babicka can search the important objects in text and their relation and subsequently to represent the relation by using graph principle. The important objects are nouns and defined another words. The Babicka system is implemented only for Czech language.
INSTREET System
INSTREET allows to pinpoint a location where a photo was shot by comparing architectural details of the example with its reference database. Just like textual text engine – it provides a list of results the user free to browse through. It was tested on an areas as large as 135 km^2. The result (median) is 0.05%, regardless of the size of the database. It means, that in median you will have to browse through 0.05% of your database to find your location.
Criminal Complex Analysis Platform
LINK is a complex analysis platform with rich features concerning data import. Unification and transformation. Mammoth is a tool for pattern discovery analysis. However it lacks advanced features concerning data loading. The integration allows Mammoth users to fetch data imported in LINK application. The integration is done using a shared-database approach.
INACT System
INACT is a software suite designed for law enforcement, experts, non-governmental organizations and all other entities involved in combating of Child Pornography. INACT software is divided into a set of two applications and a database module utilizing the Query by Example (QbE) approach.
FERRET – System for Searching Textual Data
FERRET is a system designed for searching textual data and social media. It allows creation of machine learnt patterns to be created and stored as profiles. The system permits real-time matching of new tweets against stored profiles to generate alerts. The profiles are learnt automatically from tweets and webpages that are marked as interesting by the user. FERRET employs SVM as the learning engine with a novel algorithm for feature selection and query expansion that achieves high precision and high recall.
CrimNet Tool
The CrimNet Tool is a web-based application to provide support for the process of managing information about organized crime gangs and monitoring their activity. The tool enables security forces to detect occurring threats and concentrate the resources on satisfying current needs.
AbcLog Application
Abclog is a desktop application supporting analysis of the log files. The main functionality includes automated retrieval of the IP information from the Internet services such as whois servers. In version 0.3 of the application operates WhoisIp (www.who.is) DnsStuffCom (www.dnsstuff.com) and RipeNet (www.ripe.net) services. It handles logs in Common Log Format, Combined Log Format, csv, xls and xlsx formats. AbcLog works on Windows (XP, NT, 7, 8) operating systems with Internet connection.
Indexing System – WWW Index
WWW Index is an indexing system for World Wide Web resources. It crawls data from web resources, indexes them, and stores the Web page content in databases. Furthermore, it can be used to offer various services based on the data stored in databases, like advanced searching and changes detection. It works properly for Polish web resources.
Digital Forensic Tool – PyFlag/FLUSH
PyFlag/FLUSH tool is an extension to well-known PyFlag digital forensic tool for discovery of application logs in the mounted file systems. The goal of the extension is to provide functionality of discovering images that are similar to the provided example. This can be helpful in detecting illegal material related to child pornography. The extension utilizes INACT for images comparison.
TouchGUI for Criminal Analysis
TouchGUI is a prototype of the touch-enabled desktop for criminal analysis. The realized demonstrator shows how such desktop can be constructed (widget-based concept allowing for extensibility and flexible when size of the workspace is concerned), what features it should include (graph, timeline and details visualizations) and how it can be operated using fingers (drag-and-drop, handling widgets, visualization control).
Blog Analyser
The developed system is designed for ormalizi the blogosphere, taking one of the largest polish blog portals – salon24.pl as an example. The analysis is based on the identification of groups of strongly interacting users. The formalism of users may have different characteristic features, so they can play different roles in communities they belong to. The system allows to identify significant roles played by users. Blogosphere is strongly dynamic so the analysis takes into consideration both evolution of groups and change of roles played by given users.
Group: SECURITY OF DISABLED PERSONS
Recognition of Traffic Lights
Traffic lights recognition in video sequences. The solution allows to detect traffic lights in real time. Software color filter is used to find candidates for traffic lights. Those candidates are then classified with machine learning algorithms and false alarms removed.
PC-based Blind Support System
Linux-based blind people assistance system supports people with decreased visual abilities. System uses eyeglasses with build in smart camera, intelligent controller with sensors and linux-based computational unit to understand scenery in front of individual. It provides information in form of sound messages about traffic lights, arriving city buses line numbers favourite places and different obstacles on the way.
Android-based Blind Support System
Android App supports blind people in their everyday duties.
Group: IMAGE ANALYSIS FOR SECURITY PURPOSES
Eye Localization in Face Images
Eye detection algorithm is a useful tool for normalizing and aligning face images. The position of eyes is found with use of neural network regression.
Automatic Plate Recognition System
The plate recognition system consists of system nodes placed around the city. The aim of each is to monitor the traffic and recognize every vehicle’s license plate. The nodes communicate over the narrow band wireless network. They can communicate with each other and pass the messages within network.
Panorama Visualizer for Fast Video Search
The demonstrator presents panorama based visualization and fast event search method for PTZ camera video. It generates a cylindrical (panorama) scene model and creates the video sequences containing selected panorama parts for a fast video search and convenient automatic and/or manual video analysis.
Stream Optimization
A stream for a camera is compressed by changing frame rate, resolution and compression. The obtained stream can fit a link capacity available in the network. Moreover, for given bandwidth the system is able to generate perfect stream by dropping some frames.
Face Recognition Module
The Face Recognition algorithms were designed to enrich the SSM2/SSO2 station. The main discussed scenario is picking some individuals from the crowd. They can be for example kind of criminals or people unwanted in some particular area. The main problem during the design was minimization of the false-positive detections. When system is aimed to pick somebody from the crowd it has to deal with big number of examples from negative class, while the example from positive class appears very rarely. One can see, that the system should have very high accuracy not to produce false alarms every five minutes.
Group: PROTECTION FROM DIGITAL THREATS
Intelligent Integrated Agent-based System
INDECT-WP3 Portal is an information portal providing a summary (including multimedia content) on all products realized within INDECT-WP3 work group.
A Platform for Analysis of Malware
Group: UNMANNED AERIAL VEHICLES (UAV)
UAV Burzyk
Burzyk UAV can be divided into small parts and transported in a backpack or a vehicle trunk. The longest element is 80 cm long and the assembly time takes up to 10 minutes. Thanks to its compact size Burzyk can be launched from hand, catapult or specially designed cart, and the landing is performed on its chassis. It is equipped with an optical or thermal gimbal and it sends live video straight to the ground station. It is also equipped with onboard computers. The most important ones are mission computer and image processing computer.
UAV Ground Station
UAV Ground station is responsible for running the UAVs’ applications and managing the UAVs’ missions. Ground station consists of:
Rugged computers with installed software for UAV operator.
Communication mast
Visual landing system hardware
UAV Autopilot
The primary function of an autopilot component is to control elevator, rudder, ailerons deflection and throttle setting in order to autonomously steer the aircraft. Such a functionality requires direct actuators control and the autopilot becomes a crucial component required to perform the flight. In current autopilot design this control can be switched to common RC radio control at the electronic level. Thus an autopilot module also includes the possibility to allow the human-controlled flight when necessary.
Return – Application Server
It is a central point managing the flow of communication in WP2 subsystems. Return is a modular approach for developing server applications. A core return system is responsible for loading, running and managing running modules that provide functionalities. It also provides methods for remote clients to connect to the server, authenticate and be authorized to access modules. Every module can provide functions accessible from other modules or from remote clients. What, when and how is available depends on the module and access configuration include in Return.
UAV Mission Application
The mission subsystem corresponds to the deliberate layer in the three-layer architecture for robotics software. Its purpose is to:
facilitate accepting high-level description of UAV flight goals
convert these goals into simple actions that can be understood and executed by the executive layer (i.e. guidance)
and request execution of these actions in the appropriate order so that the flight goals are fulfilled in a potentially unsupervised manner.
Group: INDECT INTEGRATION, DATA and PRIVACY PROTECTION
Integrated Access to Services
The Integration System (IS) enables access to services to end-user. Integration includes the following layers: physical (data source) level, data processing level, high-level service (analysis, decision making). To achieve scalability, the system for integration does not run any tasks other than tasks for integration, logging of client requests and end-user authentication. The IS has no direct access to data sources, and it calls remote services. The IS has functionality of User Interface. End users can define a set of functions offered by the UI.
Prototype of INDECT Portal Document Management Subsystem
INDECT Portal Document Management Subsystem (DMS) is a Web application that supports police officers reacting to crisis situations in their online-represented decision making. The system coordinates the workflow of actions performed by the involved officers, the access to related documents and remote INDECT services in accordance to predefined procedures. The system is designed in a manner that allows to control how it is being used by portal operators what significantly reduces the likeliness of the system’s misuse.
Integration Platform
The Integration Platform is a command and control system which interacts with different Indect applications. The basic functions are event notification and surveillance for operators. The notification is about events, which are detected by Indect WP7 algorithms. Surveillance is done by audio or video streams delivered by node stations of WP1
Node Station and Central Station Framework
A modular system for data acquisition, processing, and transmission. It allows to create a data processing pipeline adapted to user’s needs. It is used as an audio/video/and normalize signals processing framework, both by developers and by end-users. The framework is platform independent (Windows, Linux, Mac, smartphones) and algorithms are developed to be mutually interconnected and exchange data in parallel. Examples of parking lot area monitoring, outdoor monitoring, bank area monitoring and other were presented.
IVAS – Interactive Video Audio Service
System IVAS is communication system which allows managing the communication for defined area of participants (end users). The central manager with web application can manage all text, audio and video communication for all end users. It means
• To manage the call between all end users. Manager can connect two end users, can establish conference call, and can send report to all or group of end user by using technology text to speech.
• To manage video stream from end user to center or opposite
• To manage video file form end user to center or opposite
• To manage text information from end user to center
Node Station based on ADTF integration Platform Communication
The Communication between a central integration platform, developed by PSI, and the BUW node station is based on an ADTF framework. For data acquisition and image processing, algorithms can be integrated into the framework by self developed dynamic linked libraries. An integration platform user is able to interact bidirectionally with the BUW node station. Different kind of events can be requested, streaming of videos can be controlled and a certain set of parameters (pan, tilt, zoom, kind of report) can be changed.
Social Threat Monitor
Social Threat Monitor (STM) is a novel Web system for documentation of user-defined threats. It contains a Relational Database and user interface based on a GIS component. The description of threats can be provided by all registered users, so the data acquisition is performed in a distributed manner through Internet. Threats are presented on a map. Combined relational and related to area search and filtering of threats is implemented.
Bluetooth Authorization and Detection
A module dedicated to detection and identification of Bluetooth devices. It allows for recognition of authorized users, and for proximity assessment. In presented scenarios loss of a signal indicates theft or abduction of a protected goods/person.
Detection of Secret Message in Static Images by Universal Steganalysis
An image steganalysis module for detection of secret message was presented. It is based on extraction of statistical features from image database (includes cover and stego images in JPEG image file format) and it also includes creation of model using classifier. Subsequently, trained model allows detection of secret message that was inserted using popular image steganographic methods.
Malicious Image Tampering Identification System
Malicious image tampering identification system (MITIS) is a state-of-the-art solution for active protection of digital images against forgeries. It allows for accurate identification of image regions affected by unauthorized content modifications. One of the most important features is the capability to restore the original content of the tampered fragments. The reconstruction procedure is highly reconfigurable, and features several quality levels. The tampered fragments can be restored when even up to 67% of the image area becomes tampered. When implemented directly on the acquisition device, provides efficient, state-of-the-art protection for digital evidence.
INDECT Block Cipher
One of the main goal of WP8 work was development of cryptographic solutions to ensure data confidentiality and privacy. During the exposition, the crucial achievements will be presented – software and hardware implementations of cryptographic algorithms. Mainly, the practical applications based on a new symmetric cipher – INDECT Block Cipher (IBC). Also, other techniques such as stream ciphers or hash function will be shown. Additionally, some other solutions (i.e. error detection methods) which increase the security will be presented.
Potential Impact:
INDECT project was performed in a close cooperation with End - Users, mainly Law Enforcement Authorities from different European countries. The cooperation enabled for efficient practical impact of research done in the project.
End Users Requirements Board (EURB) was established as an advisory body of the project. Representatives of the following entities were engaged in the EURB activities:
Police Service of Northern Ireland
Polish Police
Czech Police
Guardia Civil (Spain)
Police of Latvia
Police of Malta
KEMEA - Ministry of Public Order and Citizen Protection in Greece
Polish Platform for Homeland Security
Romanian Police
A number of INDECT solutions were successfully tested by Police forces. An example is the application for combating Child Pornography developed within the INDECT Project was tested by the INDECT partner - GHP. The interest for testing this application was expressed by the Czech Police (which was not part of the INDECT consortium). An agreement for testing of the INACT application between INDECT Coordinator and Czech Police was prepared. The application was delivered and installed for Czech Police for testing.
Another exploitation of the project results was a project dedicated to design an application and an intelligent system supporting people with sight disabilities.The new project benefits from basic methods developed by INDECT like image recognition (e.g. bus numbers) and sound recognition (speaker’s announcements) which are most applicable for people with sight disabilities and the blind.
The know-how from the INDECT project is going to facilitate the final product development. The techniques of project coordination and deployment will improve the new project cycle.
As a follow-up of INDECT as the research project, it is planned to set-up a spin-off company in order to commercialize prospective results of the project. For example, al support from the AGH was announced as concerns establishing a spin-off company in collaboration with the AGH’s Academic Incubator of Enterprise in order to set up the new venture.
The licensing will be commercial with some potential licensing at no charge for research and the University is taking the leading role as the INDECT project coordinator.
Solutions elaborated in INDECT constitute a foundation for creating new tools and prototypes that enhance security of citizens. These technologies, including automatic threat detection, watermarking and privacy and data protection, can be reused by companies in this domain.
The theoretical knowledge and lessons learnt from the project are shared with the students and are the valuable input to the content courses and seminars. Additionally, the students get the unique possibility to actively participate in test scenarios and have chance to validate the theoretical knowledge in practice. The research staff participating in the project benefit from the exchange of knowledge with other European partners which also positively impacts on other research projects. It should be noted that the networking build on this project leverages further across European collaboration in new research initiatives. The theoretical results of the project are being disseminated to the scientific community at the special conferences and events organized by INDECT partners.
Societal Impact of INDECT
Analysis of Societal Impact of the project was performed by an external expert in this area, Peter Burges, Research Professor at the Peace Research Institute Oslo (PRIO) and Senior Researcher at the Institute for European Studies at the Vrije Universiteit Brussel. He is Editor of Security Dialogue, an international scholarly journal dedicated to innovation in security research and Series Editor of the Routledge/PRIO New Security Studies collection.
Societal security needs addressed by the research
The INDECT project responded to the FP7 ESRP call for 'intelligent urban environment observation system' (SEC-2007-1.201). The project summary sets out its primary technological objectives, namely:
to develop an intelligent system for automatic detection of threats and recognition of criminal behavior or violence,
to develop new methods and techniques providing tools to support activities of Police officers, including tools for threat detection in the internet; this includes the development a new type of search engine combining direct search of images and video based on watermarked contents and storage of metadata in the form of digital watermarks, and
to develop techniques for data and privacy protection in storage and transmission of data based on quantum cryptography and new methods of digital watermarking (INDECT summary).
These primary aims were based on a number of assessments and assumptions about the needs of society. The project takes its point of departure in a certain set of assumptions about the security of citizens and what it describes as the 'weak point' or vulnerability of European critical infrastructure. This vulnerability is European critical infrastructure and of
Security is becoming a weak point of energy and communications infrastructures, commercial stores, conferences centers, airports and sites with high person traffic in general. Practically any crowded place is vulnerable, and the risks should be controlled and minimized as much as possible. Controlling access and being able to respond rapidly to potential dangers are facilities which every security system for such environment should have. One of the best ways to tackle such problems is through active observation. This explains the real need for intelligent monitoring and information systems in urban (and wider) areas (INDECT summary).
The project description asserts that security is 'becoming a weak point', but the nature, causes, meaning and implications of this 'weak point' are not described or in any way documented by the project. This is a particularly critical point and would give considerable force and legitimacy to the activities of the project. If this 'weak point' were documented, described and analyzed as a social phenomenon, the project's relevance would be anchored. While in a very broad way the assumption of a series of security weak points might well be thought to be reasonable, they in actuality represent a vast range of complex social, human, political, cultural and technical relations. Assuring both the project's technical success and its political support in terms of democratic accountability implies clarifying and following up on the societal meaning of these premises.
The vulnerability of 'energy and communications infrastructures' to either malicious attack or catastrophic failure is largely a technical one. These technical systems can be damaged or destroyed through mechanical causes, either through intentional sabotage or by one form of accident or another. There is a general belief that dependency of European society on such technical infrastructure is increasing. This belief, which is not detailed or documented in the project, is linked to implicit and explicit understandings of societal security and how these strengthen or weaken European dependence on critical infrastructure. Such dependence varies considerably according to local society, cultural, tradition, family, living style, economic wealth, and geographical variation.
A more thorough analysis of the societal security of critical infrastructure would also focus on the fact that terrorist attacks as a rule are not organized instrumentally in such a way as to realize the goal of interrupting the resources delivered by critical infrastructures. Terrorism is carries out symbolic acts, organizing events that create meaning, reminders of vulnerability. The aim of terrorism is not to stop the flow of electricity, to take one illustration related to critical infrastructure, but rather to create fear and unease. Addressing terrorist threat to critical infrastructure by focusing on purely technical function of such infrastructure risk missing the point of terrorism and thereby failing to protect against it.
The security of 'commercial stores, conference centres, airports and sites with high person traffic' poses a different set of challenges related to societal security. Some of these are linked to the questions of privacy and data protection that are evoked at several moments in the INDECT project description and scrutinized in detail by the 'Ethical issues' reports (D0.5 D0.6 D0.7). Other central concerns for societal security are not covered by the primary aims of the INDECT project, concerns that, if carefully accounted for, would strengthen considerable the impact of the project on the actual security of citizens, and on public perception of the value- added, relevance and accountability of the project. First and foremost, the project's basic premises do not contain a consideration of the social and cultural dimensions of public spaces in which the systems and measures developed and proposed by INDECT will deployed and expected to function and bring security results. Public spaces different considerably in their social organization, and thus in the way that dangers are perceived and played out them. A shopping mall, a railway station and a public square, for example, differ greatly in terms of both the objective safety of those who circulate their and also in terms of the ambiance of security and vulnerability, that different urban spaces create and afford. There is, in addition, numerous nuances and distinctions between such public spaces in different cultural, social and geopolitical situations. The difference between small cities and large, southern and northern cities, coastal or plane areas, rich and poor, arid or humid, etc. Furthermore the security needs and demands of citizens in general vary greatly from Member state to Member state, from region, from one economic class to another, according to language, family, employment, etc.
In some cases more technological security measures of the kind proposed by INDECT enhance security. In other settings, more technological security actually lowers security, causes insecurity, etc. While it is true that many crowded places are vulnerable, it is not always for being crowded that they are vulnerable. Nor is it clear to what degree this insecurity stems from the crowdedness or not. In any case, the 'risks' to which the INDECT description refers are in all likelihood highly complex and socially differentiated as described above. The assumption of a 'need' to reduce such risk is neither substantiated by social risk research, nor politically justified, even if such a justification and legitimization would be easily obtained. The public acceptance of security measures that border on interventions into public space or the private sphere would likely be much stronger if these questions of social compatibility were dealt with and publicized accordingly.
Similarly, 'controlling access', will have both societal costs and benefits when it comes to the security of citizens. It will be a viable tool for the protection of citizens against threats and dangers only under specific, socially determined conditions, that is, as a function of how movement and flow is structured and how it flows beside, though, and around potential dangers. The value of access will be display differing characterizes in differing settings, and the social traditions and institutions that assure the flow of citizens in and through liberal market models.
By the same token 'potential dangers', such as they are described in the INDECT project, are only partially objective, concretely measurable, identifiable and traceable as dangers or threats that can be simply removed from society and dealt with as foreign agents. The perception of danger-like the social acceptance of the security measures designed to address danger--is socially complex, woven into understandings and experiences of technologies, security actors and institutions, and they way their presence and actions are woven into the dangers themselves.
The basic tenet of societal impact review is that security is a societal challenge. The ultimate object of all security measures is people. The primary question for societal review thus becomes: how does the reflect this awareness in its principles and procedures. While any given security challenge can most certainly be addressed by technical means, it can only be said to have met its aims if it makes people more security in meaningful and verifiable way. A detailed examination of these issues in the development phase of the project would likely have the double of effect of the making the project considerably more effect-since it would in that case actually be addressing scientifically confirmed security problems.
Approach to societal needs: From security measures to security threats and back
The security weak points set out in the basic documents for the project are ultimately societal. They have clear technical dimensions (concerning the robustness of technical systems, communication procedures, coordination, harmonization of technical equipment, etc.) Yet these technical dimensions are embedded in societal contexts, reflecting societal needs, and positioning possible solutions squarely in options provided by societal limitations, be they public fears, cultural customs and traditions, ethical constraints, legal regulations.
The security weak points identified by the project are not by any means incorrect or unreal. They are plausible from a technical point of view. The challenge lies in the fact that they are also deeply social, implying important assumptions, and considerable consequences. If these weak points are left in the discourse of technology, natural science and the hard logics that accompany them, they will themselves inevitably be fused with threat itself. Security research that does not take the consequences of itself being socially embedded with deep social linkages at all levels, will not be able to respond to insecurity, which itself is fundamentally social.
Moreover, an approach to security that revolves around technology for technology's sake not only risks not adequately addressing the insecurity of society, it also risks generating new insecurities. The old geopolitical version of the 'security dilemma' noted that nation states that arm themselves against belligerent neighbours do little more than make those neighbours more insecure, thus obliging them to upgrade their armaments, thereby increasing the overall insecurity. A parallel effect is generated by advancing security measures that do not communicate with the needs of those in society whose security is in question. The new security dilemma of our time: Blindly technological security measures have a tendency to generate new security threats as a secondary effect of protecting against the principal threats.
Threats to society addressed by the research
The INDECT project proposed technological measures and strategies for the development of technological systems to address threats of violence to urban areas, in particular public spaces, threats to internet infrastructure, tracking of objects and individuals, new technologies for digital watermarking and tracking those involved in child pornography and trafficking in human organs, tracking of individuals and transactions involved in various types of fraud. It also aims to develop internet-based tools for preventing crime and terrorism. It will develop high capacity, high speed computer tools for processing data about potential and actual crimes. In addition, it aims to create tools and technology for privacy and data protection during storage and transmission of information.
In short, the INDECT project aims to address a variety of perceived threats to society: violent, criminality, terrorism, particularly in urban settings, child pornography, trafficking and fraud. On a very general level, these threats have currency and are more or less uncontroversial. These are threats to the security of society. They interwoven with society in complex ways. Most prominently, the threats themselves are not objective, but rather are products of perceptions and values, from cultures of fear and safety, from experiences and understanding of how various dangers function in society, how they are dealt, in what ways they are increased or decreased. Criminal violence and terrorism are most certainly threats to the security of society, and yet understanding how to reduce threat requires understanding society's experience of that threat. The key to assessing the project's overall societal impact begins with the question of how the threats it aims to address are linked to societal concerns, i.e. its values, customs, traditions, insecurities, robustness, resilience, etc.
The project's aim of generating technological solutions to track objects and individuals, through advanced surveillance tools, watermarking, and technologies of identification and tracking. intend to address threats. But given that they address potential or future threats, society's acceptance of security measures and insecurity toward future threats will be clear elements in the analysis. The guiding in principle of all such pre-emptive crime-fighting is that the crimes lie in the future, that their reality is a potential, that they are something to come. Crimes and acts of violence of the kind to be handled in the INDECT project are understood through assumptions and informal processes of speculation. Unlike preventive research on hazard or crisis management INDECT is anticipatory. It targets actions that have not happened yet. This ground for this speculation of threat requires a deep understanding of society and its security needs.
Approach to threats
Not only the degree to which the fear of such threats is generated in and through society, but also the basis for developing technologies aimed at addressing such The primary principle when seeking to understand the approach to threats in terms of societal impact is that security measures have a societal correlate. Their success or failure depends to a varying degree to the character and values of the society in which they operate, and on the degree to which the measures are tailored to and respect the complexities of security in society.
Beneficiaries of the research
It is simple to imagine and in general taken for granted that the primary beneficiaries of the INDECT project are European citizens. However, it is important to examine closely the project's means and aims, and reflect over the complex relations that these have to citizens.
Much of the ethical justification for the project's aims is drawn from a relatively uncritical assumption that good will and instrumentally sound research design will bring favourable research results. This is a demonstrably inconsistent assumption. Moreover, the more a project is complex and, layered, and long, the more the path between its good intentions and robust methods that lie at its outset and actual outcomes becomes invisible for those with either curiosity or concern. This invisibility in turn generates insecurity that may interfere with or even subvert the project's very ability to contribute to security.
This additionally due to the fact that the societal world view of the experts and specialists that have designed and carry out the project is likely quite different from that of the citizens that are intended to benefit. Neither is right or wrong. Elements of both will need to be brought into the project's overall perspective in order to make valid for all. This is above all crucial when seeking to understand and then improve public understanding of and attitudes toward the project’s means and aims.
All security measures, be they in the form of technologies are human actions, tend to have secondary effects that can often conflict or come in contradiction with the aim of security. Those that are not reflected, legitimated and justified in social and cultural terms, will further detract from its ability to bring security benefits.
Privacy, to take a central example from the project, is not only a right. It is also a fundamental human need. The self, the personal sphere, the home, the family, etc. have value unto themselves. For some, they are sacrosanct. Compromising privacy, as so many of the new technologies tend to do, will weaken the benefits of privacy in way that cannot be reduced to a 'balance' between security and privacy.
An important part of the challenge involved in the work of INDECT is that the project's scope and complexity is uncommonly large. The vision of security it builds upon is broad and comprehensive, involving many actors and many interested parties. The perspectives behind the technologies it proposes to develop and implement, and links and logics that join them are far from obvious, even for internal participants of the project. This has a distinct effect citizens' reaction to the work and, thereby, to the potential security benefits it will potentially bring. The path that joins the means and ends of the project is largely invisible. And yet it is precisely this path, and not the advanced technological accomplishments alone, that will bring the benefits of security to society.
The sophisticated tools under development for the analysis of video and audio data, for example, may be envisaged to benefit citizens and groups in the long-run or on the large scale. However, from the perspectives and experiences of everyday citizens they may be experienced as intrusion, because the larger or longer term benefits exceed the perspective. It is important to cast light on the differentiation of benefits along different levels and different time-frames of the project.
By the same token, computer aided detection systems, electronic watermarking, or quantum cryptography may only provide security within a particular horizon of possible crimes or actions. Most citizens are not concerned by what happens within this picture. Even though the security measures are taken in social settings, and even though the threats they seek to address are also imbedded in society, the complexity of society and the differentiate perceptions of threat, understandings and experiences of threat, and exposure or vulnerability to threat No project can solve all security problems for all people. An awareness of what the INDECT project cannot do, and an understanding of the false securities and insecurities it can generate as side effects in society is fundamental. This is the case both internally in the framework of the project and externally in the public sphere.
Security research does not benefit everyone in society on an equal basis. Some do not benefit at all. What are the correlates of those that benefit from security measurements? What distinguishes them? : economic well-being, social standing, class, caste, family or social background, education, etc. Such properties and be simply correlated with and the relates of prejudicial benevolence.
Society as a whole as beneficiary
In this sense it must be emphasized that benefiting society as a whole is a difficult challenge. And yet it is essential to ask the overall question of societal benefit. Framework research, funded by public monies and with a considerable demand of public accountability. Society at large counts and should be considered in the justification of measures and of scaling of measures.
Clearly, INDECT will benefit society unevenly. No research project can meet the needs of society in its entirety. However, the particular inequalities of distribution of the work on INDECT should be explored and explained, including the considerable benefits that will flow to the industrial private sector. European society can, in general terms, be described. Its priorities, values, needs, and requirements can indeed be mapped and the results fed to both those who engineer the technologies at the heart of INDECT and the dissemination and communication challenges that will follow it. The overall societal benefit of the project must be explained and understood on a higher, political level. Like any research project within the European Security Research Programme, INDECT should be exemplary. It should in what it undertakes to do and also what it chooses not to do, it should embody the political notion of what is the best for all of society. This description and explanation can be put in terms of societal values.
The point of entry for such an analysis is the question of the value the measure have. What are the values that intelligent analysis of video and audio data can preserve? How can one be clear about the general value to society of search engine designs supporting semantic searches based on a watermark system? How will intelligent processing of data serve society?
In all of these communication issues, the challenge boils down to transcending the technical vision of the research, making it sensible to the principles of the EU or any free society that such research will be deployed in.
Ethical Issues
The project faced several ethical issues related to its potential impact and use. Handling of collected personal data while protecting privacy and confidentiality is of major importance for the project; therefore a specific task monitors on this aspect throughout the project. For this purpose Ethics Board was established which supported the project consortium in examining the societal, political and legal aspects of potential applications (especially dual-use applications), defined and approved the future exploitation plans of the project results, and advised in dissemination and communication strategy of research results to a wider audience.
All of the research activities within INDECT project were carried out so as to ensure the appropriate balance between the protection of the rights of the individual and the protection of society. Ethics Board had for the main object to ensure strict compliance of research outcomes with already established rules concerning privacy, data protection, to ensure genuine informed consent of all those participating in the project, and to ensure that information is only used for its intended research purpose. It was also responsible for managing and monitoring all ethical aspects of the project. These aspects included the promotion of gender equality.
The Ethics Board has a broad membership, designed to exert strict control over the project. It included representatives from data protection experts, Non-Governmental Organisations, academic world, industry and the Police Service of Northern Ireland, which is generally recognised as adhering to the highest standards of human rights protection in all aspects of its work. The Board did not view its role as ensuring compliance as a minimalist task, solely designed to ensure legal compliance. Rather, it perceived its function as broader, including overseeing scientific and societal issues related to the research activities conducted within the project.
The algorithms and methodologies underlying the project relied primarily on previously available, usually public, information sources. These include monitoring cameras, public Web pages, etc. The research covered by the INDECT grant, did not consider processing of highly sensitive material, such as telephone intercept, VoIP, etc.
The main objective of INDECT research was to make the monitoring and search process (and procedures) more automatic. This will ultimately allow for more informed decision-making. The value that will be added by deployment of INDECT research outcomes is that existing systems would operate with less human intervention, which will lower the level of subjective assessment and the number of human mistakes. This means less staff will be required for supervision of surveillance activities (e.g. monitoring of CCTV camera networks). This will resulting in less opportunities for illegitimate use of such information, or for human error to result in violations of the rights of the individual. There will also be economic benefits, in terms of the reduced staffing requirements. Police officers could be freed up to carry out frontline policing tasks.
The algorithms that are under research are intended to be used solely in cases where this is justified. Existing legal requirements will apply where the police or other agencies seek to use tools based on INDECT research outcomes in any investigation. These tools will not, and could not, involve any reduction of existing international and national legal protections. Integrated safeguards, such as blurring, etc., allow sophisticated controls to be placed on images (e.g. car registration plates) within recorded images.
Tools based on INDECT research will enhance the ability of the police to protect the public. An example can be an algorithm for detections of dangerous situations, such as: abandoned luggage or withdrawing a dangerous object (a knife or a gun). Such tools will allow the police to target resources where they are needed. It would not involve mass surveillance, but rather targeted efforts towards detecting the real threats.
It is important to note that a person highlighted by tools based on INDECT detection algorithms would merely be brought to the attention of the relevant authorities, so that normal lawful measures can be taken. The fact of highlighting by itself would involve no legal consequences for the person, and no permanent record would be kept, unless there was a specific legal reason to do so.
Tools based on outcomes from INDECT research are going to be designed to protect the public and its use in any given situation will have to comply with national and international rules regarding privacy. In terms of its design and implementation, the EU Charter of Fundamental Rights applies. As a result of the Lisbon treaty, the EU became a party to the European Convention on Human Rights. These safeguards are additional to those already enjoyed throughout Europe.
National Standards regarding privacy (e.g. the Data Protection Act 1998, the Human Rights Act 1998 and the Regulation of Investigatory Powers Act 2000 in the United Kingdom) will also apply in every EU country.
The sentence: “if you have done nothing wrong, you have nothing to fear” is only true if every aspect of the criminal justice system works perfectly, on every occasion. Tools based on INDECT project research outcomes will provide EU Member States with the technology to ensure that decisions around public safety are based on the maximum amount of relevant information available.
List of Websites:
www.agh.edu.pl/en
Coordinator:
Prof. Andrzej Dziech
Tel: +48 (12) 6172616
Fax: +48 (12) 6345582
E-mail: dziech@kt.agh.edu.pl
AGH University of Science and Technology
Al. Mickiewicza 30
Poland – 30-059 – Krakow
The EU-funded project “Intelligent information system supporting observation, searching and detection for security of citizens in urban environment” (INDECT, http://www.indect-project.eu/) worked on developing solutions designed for police and other law enforcement authorities in the EU. Focusing on automatic threat detection in urban environments, the project team developed a set of tools supporting decision-making in counteracting threats and criminal activities.
European scientists and researchers developed solutions and tools for automatic threat detection. The primary objective was to develop advanced and innovative algorithms for human decision support in combating terrorism and other criminal activities, such as human trafficking, child pornography, detection of dangerous situations (e.g. robberies) and the use of dangerous objects (e.g. knives or guns) in public spaces. Efficient tools for dealing with such situations are crucial to ensuring the safety of citizens.
A significant part of the project was dedicated to the development of tools and methods for data and privacy protection. The processed information is protected before its transmission or storage to prevent any attempts at unauthorized access. Dedicated tools are being developed to protect citizens’ privacy in areas covered by visual monitoring systems. The main tasks of INDECT can be divided into 3 categories as shown below:
The first category, “Intelligent Monitoring for Threat Detection”, is dedicated to the automatic detection of real world threats (video and audio analytics).
In the second category of “Threat Detection in Computer Networks” (Internet Forensics) the research focused on identification of sources, which distribute child pornography, or information related to the illegal trade in human organs. Project members developed software in order to combat organized crime. One of the main tools developed within this category is INACT, a system for rapid detection of child pornography on storage devices (mainly hard drives) captured by police forces on the basis of a warrant. INACT is especially useful if the illegal content is hidden in large data volumes.
The third category was focused on the development of “Data and Privacy Protection” techniques. Project members elaborated on data and privacy protection techniques using digital watermarking and cryptographic algorithms.
The main tangible INDECT results are over 50 working prototypes. Some of them were presented and awarded at international fairs and conferences, including:
Gold Medal at Brussels Innova 2010,
Gold Medal at Poznan International Fair 2010,
Gold Medal at Technicon 2010,
Gold Police Star at Europoltech 2011,
Gold Police Star at Europoltech 2013,
Silver Police Star at Europoltech 2013.
Gold Medal at International Concours Lépine 2014”, Paris
Award of Polish Ministry of Science and Higher Education)
Two gold medals at BRUSSELS INNOVA 2013
The Gold Police Star and The Silver Police Star Award at Europoltech 2013, Warsaw, ”SUPERMODERN 2013” Programme
INDECT research teams published over 400 scientific papers - also in such prestigious journals as “IEEE Transactions on Information Forensics” and “Security and IEEE Transactions on Image Processing”.
INDECT organised annual international MCSS conferences in 2010, 2011, 2012, 2013 and 2014.
Other INDECT activities related to dissemination activities comprised:
Meetings with Members of European Parliament
Meetings with Members of National Parliaments
Participation to events related to privacy and data protection such as “Computers, Privacy & Data Protection” Conferences held in Brussels yearly and yearly working meeting with Polish Members of the European Parliament organised by Data Protection Officer.
INDECT team actively cooperated with bodies responsible for personal data protection, i.e: Polish Inspector General for Personal Data Protection, Article 29 Working Party and NGOs (such as Panopytkon).
Project Context and Objectives:
The Internet and globalisation have brought the world closer in many ways, but they have also empowered international crime rings and terrorist groups. Sophisticated theft, child pornography, terrorist attacks and human trafficking have taken on an alarmingly global dimension, prompting a need for high-tech solutions to neutralise such threats.
The EU-funded project “Intelligent information system supporting observation, searching and detection for security of citizens in urban environment” (INDECT, http://www.indect-project.eu/) worked on developing solutions designed for police and other law enforcement authorities in the EU. Focusing on automatic threat detection in urban environments, the project team developed complex software to support decision-making in combating criminal activities and thwarting terrorist attacks.
INDECT project involved European scientists and researchers in the development of solutions to and tools for automatic threat detection. The primary objective was to develop advanced and innovative algorithms for human decision support in combating terrorism and other criminal activities, such as human trafficking, child pornography, detection of dangerous situations (e.g. robberies) and the use of dangerous objects (e.g. knives or guns) in public spaces. Efficient tools for dealing with such situations are crucial to ensuring the safety of citizens. A significant part of the project was dedicated to the development of tools and methods for data and privacy protection. The processed information is protected before its transmission or storage to prevent any attempts at unauthorized access. Dedicated tools are being developed to protect citizens’ privacy in areas covered by visual monitoring systems.
INDECT - Main Research Areas
The main tasks of INDECT can be divided into 3 categories as shown below:
The first category, “Intelligent Monitoring for Threat Detection”, is dedicated to the automatic detection of real world threats (video analytics). Contemporary Closed-Circuit TeleVision (CCTV) monitoring systems are mainly based on the continuous observation approach, where the monitored area is under constant supervision by the operator. If the operator is dishonest, this can potentially lead to privacy abuse. INDECT improved CCTV strategies to focus on potential threats rather than on continuous monitoring of day-to-day activity. The system automatically alerts operators in case a threat is detected, enabling them to make the final decision on whether to react in case of a real threat. The INDECT project adopts a new approach based on threat monitoring (also known as “black/dark screen monitoring”). In contrast to the traditional approach, it focuses on potentially threatening situations, and gives the operator access to the video only when his attention is actually needed. The system automatically detects dangerous events and alerts the operator, who makes the final decision about the action that needs to be taken. The process is carried out in accordance with all relevant regulations. The research conducted within the project aimed to improve the efficiency and accuracy of existing video monitoring systems. INDECT was not installing any cameras in the EU; nor was it filming people at random. It was not connected to any existing database or social network.
In the second category of “Threat Detection in Computer Networks” (Internet Forensics) the research focused on identification of sources, which distribute forbidden content (such as child pornography), or information related to the illegal trade in human organs. Project members developed software in order to combat organized crime. One of the main tools developed within this category is INACT, a system for rapid detection of child pornography on storage devices (mainly hard drives) captured by police forces on the basis of a warrant. INACT is especially useful if the illegal content is hidden in large data volumes.
The third, final category was focuses on the development of “Data and Privacy Protection” techniques. Project members elaborated on data and privacy protection techniques using digital watermarking and cryptographic software. This would help combat malicious tampering with multimedia, improving security of data transmission and storage. The system was designed to protect against the exploitation of sensitive information such as license plate numbers or faces. There are two main research directions related to this category – development of digital watermarking, and cryptographic algorithms. The digital watermarks are used to secure multimedia content against malicious tampering, and to provide a novel privacy protection mechanism. The approach adopted enables sensitive information, such as faces or license number plates, to be collected and displayed to authorized system users only. The development of new cryptographic algorithms delivers enhanced security for both data transmission and storage. A modern symmetric cipher has been designed, verified, and implemented both in software and hardware. This research also addresses the potential of adopting quantum cryptography to further improve the security of the solutions developed.
Overall, the project developed its strategies and tools with data protection and privacy in mind. Measures adopted included minimising unnecessary video recording and protecting information before it is transmitted or stored. The project’s results were highly received and demonstrated great potential in achieving their aims. The proposed technologies won several prizes between 2010 and 2014, gaining interest of end users and relevant stakeholders.
INDECT could prove particularly useful in alerting authorities about incidents on train platforms or where hooligans throw objects in stadiums, for example. This can be achieved by enhancing existing systems without adding extra surveillance cameras.
Varied benefits range from detecting abandoned luggage to analysing threatening sounds. Once exploited the solutions will support the control of cybercrime, terrorism and hooliganism.
Main Project Objectives and Results
To develop an intelligent information system for automatic detection of threats and recognition of criminal behavior or violence.
New, award-winning, methods have been developed, for tracking of objects of interest (source of the sound, or a “trigger” of visual event) by Pan-Tilt-Zoom cameras pointing automatically at given location. The location is provided by sound source detection and localization methods, or by video analysis of the scene observed by a fixed megapixel camera. Video processing has been supported by new solutions for vital problems of video quality in distributed monitoring: automatic assessment of streaming conditions and video compression artifacts by means of Quality of Experience methodology. Algorithms based on the all above-mentioned methods, together with secure mechanisms for acquisition, storage and effective sharing on demand of multimedia data gathered, have been implemented in 15 Node Station prototypes that have been developed, installed in cooperation with Partners, preliminarily tested in simulated real conditions, and prepared for extensive tests phase. Finally, the first prototype of a Central Station has been developed, serving as a management point in the distributed pilot installation of the monitoring system.
Overall self-organising computer network architecture model has been proposed. A biometric features analysis component, based on video and image information, has been proposed. Similarly, a multimodal features analysis component has been provided. Finally, a prototype of automatic event detection system has been developed.
Furthermore, proposed logical structure of the Unmanned Aerial Vehicle (UAV) systems has been elaborated. Systems architecture has been described. Proposed algorithms for positioning, tracking and predicting of the position of tracked objects have been reported. Similarly, the proposed algorithm and mechanisms for wireless transfer of partitioned spatial data have been provided as well. Also, the proposed algorithms and methods for autonomous steering and navigation of UAVs have been reported. Moreover, the proposed algorithms for mission planning for groups of autonomous UAVs have been delivered. Finally, the prototypes for the implemented subsystems have been elaborated.
To develop new methods and techniques providing tools to support activities of police officers, including tools for threat detection in the Internet
INDECT developed high precision crawler technologies navigating the Internet World Wide Web (WWW) websites, by following the hyperlinks and analyzing the content found in the websites, in order to find highly specific information on the Internet, allowing for mining for and detecting suspicious websites. These processes can be based on textual or multimedia searching.
As far as textual searching is concerned, software for learning relationships between people and organizations through websites and social networks has been developed. The software is able to learn behavioral profiles of known criminals.
As far as multimedia searching is concerned, software has been developed, allowing for integration of developed different multimedia analysis tools (called plugins). Different plugins have been integrated or are foreseen: image/graphic recognition, face detection/recognition, symbol detection, and automated child pornography detection.
In order to allow continuing investigations related to child pornography, INDECT developed Advanced Image Cataloguing Tool (INACT). The purpose of the tool is to catalogue the images found at the arrested suspect’s HDD and to compare the images against the images gathered during the investigation. The outcome of such search can be used to prove that the suspect is in the possession of the forbidden content and/or has been actively sharing the forbidden content.
For targeted criminal analysis, INDECT developed new components for the award-winning LINK analytical platform. The components allow for registration of the user operation history, executing sequences of operations on large volumes of data, and integrating selected Internet data sources (e.g. register of companies). The criminal analysis platform has been integrated with the pattern recognition application, communicating with a police officer via multi-touch-enabled analytical desktop interface. Further targeted criminal analysis applications have been developed, including: an extended version of the KASS Social Network Analysis system with the functionality dedicated to the analysis of data coming from Internet blogs, abcLog application for retrieving information about IP addresses for large volumes of data based on Internet data services as well as a system supporting management and strategic analysis of data related to criminal group activities.
Moreover, a prototype of the platform for the capture and analysis of malware has been created.
Finally, INDECT developed tools to process the information provided by citizens via emergency website.
To develop techniques for data and privacy protection in storage and transmission of data.
INDECT developed content access protection system, being a new concept of reversible image censorship, which uses a reconstruction reference embedded in the image by means of a digital watermark to restore the original appearance of the censored fragments. The principal application of this scheme is to protect privacy in video surveillance systems.
The proposed content protection scheme consists of two modules: an encoder and a decoder. The former is used just before the format conversion step which prepares the images for storage or transmission. The latter is a client-side module which, upon successful authentication of an authorized user, is capable of decoding the additional information from the watermark and performing content reconstruction.
INDECT has implemented a proof-of-concept prototype for protecting privacy in intelligent video surveillance systems. The system detects and tracks sensitive patterns (e.g. faces, car license numbers plates) and performs real-time anonymization.
Content access protection is one of the most interesting and most promising watermarking-related research topics carried out by the INDECT project. It addresses the privacy issues, which have become one of the most important aspects of the ongoing broad implementation of video surveillance systems.
To elaborate procedures and tools for making information more secure from unauthorized access and use.
All possible aspects of information security in INDECT project were defined. VPN types and technologies, analyzing their usability for INDECT project were specified.
Specification of simulator for cryptographic algorithms and a testing-tool, explaining the software to be used was done. Mathematical models as well as the GUI of components were provided.
Component for development and tests of a new Quantum cryptography protocol were delivered. A methodology, key parameters and simulation and measurement software tool for the existing and new protocols were elaborated.
Terminology of European ID authentication and identity management making an analysis of its components and giving the conceptions for Liberty Alliance, WS-Federation и SAML were defined. Subsequently recommendations for development and implementation were done as well as the relation between European ID authentication on one side and identity management and other WPs on the other.
Different aspects of Security and Authentication in Mobile Adhoc networks were specified. Blueprints of the secure multi-path routing protocol for ad-hoc networks were implemented.
New block cipher was constructed and evaluated with respect of its vulnerability to errors. Innovative encryption/decryption algorithm was implemented. Also tests and simulations have been performed to check and evaluate the security levels. The simulation software was also elaborated for evaluation and comparison of different cryptographic algorithms.
INDECT successfully passed the Final Review of the project, which was held in Krakow in June 2014 (3 reviewers from European countries took part in a panel meeting) and Ethics Reviews: including the one held in Brussels in March 2011 (9 reviewers from European Commission and from several European countries took part in a panel meeting) and the final Ethics Review.
Project Results:
This section provides a short overview of selected INDECT project research areas, results and brief description of prototypes.
Node Station Framework for Multimedia
GUT Multimedia System Department team developed an Intelligent Surveillance System for automatic event detection. Real-time and concurrent processing of audio fulfils the goal and video streams, for object detection and tracking, by geo-positioning of detected objects for automatic PTZ camera positioning, and adding Bluetooth object identification. The software was developed and tested in a dedicated Node Station (NS) framework (hardware and software solution). In the period of 2013-2014 the NS was tested in real conditions, with a focus on algorithm optimization, and user accessibility, and was integrated with other elements of the INDECT Project.
The NS system was tested in outdoor and indoor conditions, by performing staged events in a controlled environments (access limited only to informed test participants). Correct handling of data streams from video IP cameras, thermal cameras, acoustic sensors, and Bluetooth transceivers were positively evaluated. The multimedia streams processing algorithms and the client application (video and audio overview and notifications) were developed as a multiplatform software (Linux, Windows, and smartphones are supported). Documented case studies include monitoring of buildings and their vicinity, parking lots, banking room environment. Experimental results were published in journals and presented on conferences and documented in INDECT reports (D1.5 D1.6 D1.7 D1.8 and D7.7 D7.8).
Real conditions evaluation of the Node Station framework was based on two setups, Gdansk Science and Technology Park, and campus of Gdansk University of Technology. The final result is a fully functional system of Node Stations, allowing for distributed processing of multiple audio and video streams, and transmission of detection results. To disseminate the solution GUT and AGH organized a Node Station programming workshop in Krakow (section 3.4.3).
Contacts and cooperation with End-Users, including the meeting at GUT and live presentations of NS framework, resulted in a report D1.7. „Complete documentation and manuals for all classes of users of the system”. Cooperation between Partners of WP1 and WP7 allowed to define requirements reported in D1.5. „Specification of procedures for data exchange including definition of system usage with respect to the law and police regulations”. Node Station framework developed in WP1 was integrated with the result of WP7, namely the Integration Platform, aimed at notifications presentation (documented in D7.7 “Prototype of complex multimodal features detection system”).
The Passive Acoustic Radar application was extended with PTZ moving camera, and advanced video processing algorithms, resulting in a “System for acoustic and visual remote observation”, that was exhibited at Europoltech fair, and awarded Golden Police Star.
The communication with other modules in the NS Framework was finally defined. The timestamp holding mechanism during frame processing in the audio events detection module finalized. The decoder module was optimized for lower computing requirements. The front-end module was extended for parallel feature extraction, so different kind of features could be extracted in real-time and combined in resulted feature super vector. New parallel front-end module and demonstration GUI interface realized.
We analyzed the impact of the system bandwidth on the ToA and final node distance estimation accuracy and for NLoS mitigation problem, especially in case of NLoS communication with UDP, the low-complexity NLoS mitigation technique for ToA estimation has been proposed. Performance analysis of the Anchor-free Localization Algorithm with Low-Complexity Method for node Distance estimation enhancement using ToA has been done. Testing of the designed localization method by the UWB802.15.4a model for wireless channel impulse response generation in the OFFICE, RESIDENTIAL and INDUSTRIAL environment. Modification of the reactive MANET routing protocol named dynamic source routing protocol (DSR). We integrate possibilities to use opportunistic routing of the messages in the case if the routing protocol cannot find communication paths. Implementation of the designed selection the best candidates nodes methods and testing of the proposed algorithm in the OPNET modeler simulation environment. Our research activities were focused on the publishing of the collected results about localization of the mobile terminals and temporary disconnected mobile ad-hoc networks (MANET) in research conferences.
Identification and Observation of Mobile Objects in Urban Environment
This part of the research concerned the Monitoring of Physical Objects and Detection of Threats and mainly related to detection, recognition and prevention methods deployed in urban areas.
The initial research activities were mainly concentrated on creation of self-configurable ad-hoc sensor network architecture, which will be able to detect many features of objects simultaneously. The issue of limited capabilities in autonomous target searching for targets with defined characteristic has been solved by applying modern methods and algorithms for image processing for automatic locating and tracking of targets.
Work in all areas has been continued focusing on network centric system, especially on multipurpose city sensor. We have been working on an algorithm for redetection a visually tracked object after a period when it is occluded. We have been also testing and analysing communication algorithms in MANET, based on Time of Flight estimation, namely MUSIC, ESPRIT and Matrix Pencil. We have been also performing test flights of Indect UAV in order to gather image data and to test available stabilization and tracking algorithms.
The following tasks were dedicated to the construction of prototypes of sensor network built from - master sensor station (communication backbone), observation sensor station (plate recognition) and mini tracking sensors. We have been also working on different techniques for LoS/NLoS (Line of Sight/NoN Line of Sight) channel identification and the ToA (Time of Arrival) node distance estimation.
In the next phase research concerned the possibilities of the Time of Arrival estimation from the channel impulse response (CIR) of the narrowband systems. The research has been also focused on the issues of multipath routing in opportunistic networks. Emphasis was also placed on the security and trust models for opportunistic networks. The results were published in the Journal of Electrical Engineering and in Oradea. Furthermore extensive research on the recognition system has been conducted. This was completely new chapter of research in this field. We have also created series of schematics, revisions, devices and casings for sensor network.
Other research activities were done in the following areas:
node distance determination enhancement by using Time of Arrival estimation in the case of Non Line-of-Sight communication with Undetected Direct Path (NLoS-UDP),
preliminary tests of sensors - filters that evolved with use of the Genetic Programming used in video-sensor station.
Design and development of a tool for monitoring Wi-Fi hotspots to detect unauthorized access and unusual or suspect behaviour
We have designed a method to authenticate Open Wireless Networks to clients. The proposed solution is a defence against any kind of Twin Evil attacks on an Open WLAN. The idea is that clients can trust the network they want to associate with without the fear of being spied or infected directly by an attacker who makes a Twin Evil attack. The authentication of the Open WLAN is based on certificates delivered by a Certificate Authority. The solution permits the authentication of the wireless networks, which avoids association with a fake AP. It also permits to encrypt the communication between a client and the associated AP to prevent passive eavesdropping. We have implemented the proposed method on Linux. To implement the solution on the AP, we use a computer with Hostapd and Compat-drivers on the computer of the client.
The participation of UC3M in WP3 is mainly related to the development of the INDECT Lawful Interception Platform (ILIP).
ILIP is a novel platform for the Lawful Interception of data communications that will enable European Police forces to monitor in a cost-effective way the Internet activities of criminals, which increasingly employ these communication networks to, both, coordinate crimes in the real world and even commit crimes directly over the Internet.
Two types of elements compose the ILIP platform:
ILIP Capture Stations. These probes are placed at the Point of Presence (PoP) of an Internet Service Provider (ISP) and capture all the traffic from the clients of the ISP covered by a valid DWW. Seized data traffic is digitally signed, time-stamped, compressed, and encrypted before being sent to the decoding server assigned to the case.
ILIP Decoding Server. Each investigation case has an associated Decoding Server that, from the packets captured by the monitoring stations, reconstructs the communication flows, and decodes the contents being exchanged by the suspects under investigation. The decoded contents will be indexed and permanently stored in a database to ease its access by authorized Police Analysts.
The development of the ILIP platform has been completed in 2013, covering both elements of the ILIP platform. Moreover a demonstration video that shows the main features of ILIP and the developed prototypes has been recorded during the first months of 2014, in order to be shown during the Final Audit to be held on Krakow on the 13th of June.
Details about the ILIP user interface and the protocol between the ILIP decoding server and the remote pre-classification plugins are described in deliverable D3.9 “Summary report on interfaces, components and implemented algorithms” that UC3M has co-authored.
We have designed and tested a new password attack for the chosen language environment. The algorithm uses newly built character frequency tables that allow testing the strength of passwords derived from the chosen language. This approach assumes that passwords are constructed as meaningful combinations of letters. This is very important, since for example testing WPA-PSK passwords is very computationally intensive so efficiency is necessary.
The new statistical approach used for attacking passwords was successfully tested for desired language and also for desired language group (for example a language model of Slovak, Czech and English words) and best practice to create very effective password candidate lists for specified language group proposed in the journal paper.
Next, the new statistical approach used for attacking passwords in specified language was successfully evaluated and a best practice to create a very effective password candidate lists for specified language presented on IEEE IWSSIP conference. The research in the area of statistical attack for specified language continued by analyzing textual databases and analysis of different language models.
New statistical approach used for attacking passwords used in computer networks was successfully implemented and best practice to create very effective password candidate lists for specified language proposed in the paper.
A journal paper named “Security Audit of WLAN Networks using Statistical Models of Specified Language Group” published in Journal of Electrical and Electronics Engineering.
A paper named “Statistical Models Based Password Candidates Generation for Specified Language Used in Wireless LAN Security Audit” presented on 20th International Conference on Systems, Signals and Image Processing in Bucharest. The language models evaluation for different domains.
Social Threat Monitor (STM) is a Web system incorporating a relational database and GIS component for deregistration of user provided data on threats. In contrast to typical Crime Mapping Systems — where only authorized services, such as Police, are allowed and responsible for selective threat reporting — in case of STM all registered citizens can input threat data of interest. Registered data can be spatially visualized allowing easy browsing of reports. Wide, distributed and unlimited access makes this kind of service a new quality on the way to threat monitoring and safety improvement. However, due to huge volumes of complex, multi-aspect data, automated knowledge management appears indispensable. A paper outlining the ideas for a Rule-Based System for report processing and knowledge management has been submitted to MCSS 2013. A new taxonomy of rules for complex knowledge management tasks was put forward. The rules are used to assess report admissibility, access restriction, perform basic processing and threat inference. Special ECA-type rules provide immediate reaction to dangerous situations and run statistical analysis of gathered data. The rules are written in Prolog.
The research started to investigate possible extension of the STM front-end towards mobile, context aware applications have been continued. With a rapid evolution of mobile devices, the idea of context awareness has gained a remarkable popularity in recent years. Modern smartphones and tablets are equipped with a variety of sensors including accelerometers, gyroscopes, pressure gauges, light and GPS sensors. Additionally, the devices become computationally powerful which allows real-time processing of data gathered by their sensors. Universal access to the Internet via Wi-Fi hot spots and GSM network makes mobile devices perfect platforms for ubiquitous computing. Although there exist numerous frameworks for context-aware systems, they are usually dedicated to static, centralized, client-server architectures. There is still space for research in a field of context modelling and reasoning for mobile devices. A novel, lightweight context-aware framework for mobile devices that uses data gathered by mobile device sensors and perform on-line reasoning about possible threats, based on the information provided by the Social Threat Monitor system developed in the INDECT project has been put forward. These concepts were put in a draft version of a paper for the MCSS 2013.
The server dedicated to hosting the Social Threat Monitor (STM) undergoes the usual maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The prototype of the INDECT system is now stored on the vigil server [vigil.ia.agh.edu.pl]. The hardware configuration of the server was given in the former report and has not changed. An improved access control policy and its implementation was reported in the last report. It was implemented on vigil, using the HTTP server. It limits the access of the service to dedicated INDECT users only.
Current maintenance of the vigil has been continued. It included periodical software upgrades, log analysis and rotation. The server dedicated to hosting the Social Threat Monitor (STM) undergoes the standard periodic maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The system runs on the dedicated Debian squeeze server, called vigil1 that provides the set of necessary tools that are relevant from the system point of view.
At least the following set of security measures need to be in place for any software described or produced within WP4: (1) the use of the software in a secure environment, (2) secure management of data sources by encrypting the used knowledge bases, (3) secure access policy, (4) secure storage of access logs and (5) anonymisation of Ids (email, URL, username etc) from data.
Combining confidence-rated user supplied knowledge from diverse sources and integrating
Work carried out within the University of York (UniYork), UK. In our method of text classification developed within Deliverable D4.12 the system takes as input the labelled data, extracts sub-sequences from the labelled data as features, and then learns a classifier using the features. The learned classifier is then used to predict the label for new unlabelled data.
The major functionality of the software is:
To provide user with a central interface to manage data of different types. For example, organ trafficking, illegal weapon trading, protests etc.
To provide user with an easy to use interface to collect data from different sources, e.g. from text file in a computer or crawl data from World Wide Web.
Use labelled data to train a classifier which can be further used to classify unlabelled data instances.
Allow user an easy mechanism to add, remove and modify the data.
Show suggestion if the classes of any of the labelled data needs to be changed. For example, if text content is labelled as organ-trafficking by the user and the software predicts that the content is normal and does not represent organ trafficking, then the software suggests such changes to the user.
Display data and output in an interface that supports search.
The system has been improved following extensive end-user consultations with PSNI. We have studied the user requirements carefully and realised that substantial effort had to be given to data collection mechanisms as opposed to data classification mechanisms. Thus, an active “human in the loop” system was devised resulting the FERRET data collection and classification engine. The prototype FERRET system is currently targeted for precise retrieval of data from social media sites such as Twitter but can be easily re-targeted for generic websites or other specific sites such as forums etc. The FERRET system takes into account the currently data gathering mechanisms and seamlessly integrates this within the data classification pipeline.
In the first stage, the system allows a case or profile to be constructed. Each case consists of a number of pattern identifiers e.g. “Kidney” and “Eyes”.
Each pattern id is defined in a separate “Patterns Section” shown below. This section permits both positive and negative keywords along with geo-location, diameter and sender id tags.
Once a list of patterns and entered, the system performs live search to retrieve matching tweets. The tweets can then be classified manually using the functionality provided by the system to manually tag these as show below.
The improvement in feature extraction is by augmenting sub-sequence feature with word-clusters set provided from CMU1 as shown below. By augmenting the cluster id in the feature dictionary now we can match two patterns even if they do not match lexically but they produce an identical cluster ID patterns.
Using sub-sequence as features creates a large feature dimension. Our experiment showed that the large feature dimension decreases accuracy. Thus we improve our earlier system by using a feature selection algorithm that identifies the features with minimum impact on classification and discards it.
We calculate F-score of each feature to measure the discriminating power of the feature. The larger the F-score is, the more likely the feature is more discriminative. We remove the feature with lowest weights and re-train the classifier. If the accuracy for the new classifier is increased, we keep the current feature set or else we retain the old feature set. The above process is repeated till there is increase in accuracy by removing features.
The accuracy of the system is significantly improved by employing feature selection and adding word clusters as features.
The software uses the trained model to generate keywords selected from the features with the highest weights during training. These keywords are provided as query to twitter. A search is performed a regular intervals and the results displayed in the table as shown below. Classification is also performed in the extracted Tweets to indicate whether the Tweet is positive or negative. The user can override these. These annotated examples are passed on to the earlier parts of the pipeline to continuously improve the classification accuracy.
Web System for citizen provided information, automatic knowledge extraction, knowledge management and GIS integration
The work described below was carried out at the AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, Department of Applied Computer Science – the GEIST Research Group: GEIST [http://geist.agh.edu.pl/].
The overall work concerned Social Threat Monitor (STM) – a GIS-based system for management of knowledge about threats provided by citizens (registered as system users). In the reporting period it was directed towards three main directions: knowledge management with rules, mobile context-aware applications and system maintenance.
Social Threat Monitor (STM) is a Web system incorporating a relational database and GIS component for deregistration of user provided data on threats. In contrast to typical Crime Mapping Systems — where only authorized services, such as Police, are allowed and responsible for selective threat reporting — in case of STM all registered citizens can input threat data of interest. Registered data can be spatially visualized allowing easy browsing of reports. Wide, distributed and unlimited access makes this kind of service a new quality on the way to threat monitoring and safety improvement. However, due to huge volumes of complex, multi-aspect data, automated knowledge management appears indispensable. A paper outlining the ideas for a Rule-Based System for report processing and knowledge management has been presented at MCSS 2013 conference. A new taxonomy of rules for complex knowledge management tasks was put forward. The rules are used to assess report admissibility, access restriction, perform basic processing and threat inference. Special ECA-type rules provide immediate reaction to dangerous situations and run statistical analysis of gathered data. The rules are written in Prolong.
The research started to investigate possible extension of the STM front-end towards mobile, context aware applications have been continued. With a rapid evolution of mobile devices, the idea of context awareness has gained a remarkable popularity in recent years. Modern smartphones and tablets are equipped with a variety of sensors including accelerometers, gyroscopes, pressure gauges, light and GPS sensors. Additionally, the devices become computationally powerful which allows real-time processing of data gathered by their sensors. Universal access to the Internet via Wi-Fi hot spots and GSM network makes mobile devices perfect platforms for ubiquitous computing. Although there exist numerous frameworks for context-aware systems, they are usually dedicated to static, centralized, client-server architectures. There is still space for research in a field of context modelling and reasoning for mobile devices. A novel, lightweight context-aware framework for mobile devices that uses data gathered by mobile device sensors and perform on-line reasoning about possible threats, based on the information provided by the Social Threat Monitor system developed in the INDECT project has been put forward. These concepts were put in a draft version of a paper for the MCSS 2013.
The server dedicated to hosting the Social Threat Monitor (STM) undergoes the usual maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The prototype of the INDECT system is now stored on the vigil server [vigil.ia.agh.edu.pl]. The hardware configuration of the server was given in the former report and has not changed. An improved access control policy and its implementation were reported in the last report. It was implemented on vigil, using the HTTP server. It limits the access of the service to dedicated INDECT users only.
Current maintenance of the vigil has been continued. It included periodical software upgrades, log analysis and rotation. The server dedicated to hosting the Social Threat Monitor (STM) undergoes the standard periodic maintenance. The dedicated INDECT server vigil has been under continuous maintenance over the reporting period. Al necessary software updates have been installed.
The system runs on the dedicated Debian squeeze server, called vigil that provides the set of necessary tools that are relevant from the system point of view..
A new method for robust video watermarking was introduced. It uses a new developed Error Correcting Code and watermark embedding procedure in the frequency domain. The ingesting of watermark bits is done by QIM into the DCT spectral coefficients of low pass filtered stream. This low pass filtered video is generated by Inter-frame multi-level DWT.
The core idea of this scheme is the usage of the new developed Enhanced 3D Hadamard Error Correcting Code, for the purpose of video watermarking. Instead of ingest watermark bits it, Hadamard cubes are used as a code words. The code word length was selected to 512 bits, which implies the message length of 9 bits. The error correcting abilities of this Enhanced 3D Hadamard code are extremely good, especially in the case of burst errors. It is possible to correct more than 256 errors and even if the whole code word is corrupted it is possible to reconstruct it error free. This method can correct more errors as till now it was possible with standard Hadamard code.
The results are very promising. In an HDTV video sequence is possible to embed around 9112 bits (nearly 1,KB) watermark information per frame, which can be error free extracted after very strong compressions of the video. The compressed video has a data rate of 5Mbit/s (compression ratio 1:240). The embedded and compressed video streams show any visual degradation.
Search Engine for Documents Based on Watermarking Technology
The main purpose of this part of research was to elaborate a complete solution for searching, identifying and storing documents and multimedia contents to make the process of identifying criminals, hot objects, or just finding document, easier and faster. In addition, the high security for information flow needed by police and prosecutor offices will be offered.
The key objectives are indicated below:
design of new algorithms both for metadata storage in multimedia content (main aims: high encryption and redundancy whilst preserving perceived quality), which utilizes a watermarking-based approach with a background in current standards i.e. MPEG-7 and MPEG-21) and fast algorithms designed for searching such compressed and encrypted content with automatic authenticity checking;
developing high capacity watermarking technology using fast algorithms of selected transforms for the purpose of fast searching;
designing and implementing the method of software assistance support for secure searching of required persons and documents, including any security activities of police and prosecutors;
facilitating the use of existing assorted repositories designed for police and prosecutor offices by adding new interfaces and integrating currently separate systems;
establishing a coherent distributed system, this assists police and prosecutors in identifying criminals and hot objects in real time;
development of a semantic search engine for local and wide urban areas including Query by Example (QBE) techniques.
An agent technology is a suitable solution for large scale and complex commercial and industrial software applications. Agents in particular can support information retrieval in heterogeneous, distributed environments, where information is stored in different formats and types of relational databases, knowledge bases, etc. Retrieval of exact and suitable information requires analysing and rejecting redundant information or restoring of incomplete data. All these tasks can be fulfilled by multi-agent systems. Multi-agents systems are structured, co-operative organisations of different types of agents.
Quality of Experience research
The publication of QoE research – the website describing no-reference metrics was created (www.vq.kt.agh.edu.pl). A set of metrics were calculated, results were collected in a MATLAB file. The paper “Key indicators for monitoring audio-visual quality” was published. Metrics were implemented in the C++ language. Required data was prepared in MATLAB’s structures. Values of thresholds for two metrics were settled. The video library was created; it is available online at http://vq.kt.agh.edu.pl/videoLibrary.html. The first implementation of method for detecting logo was created. Implementations of method for detecting slicing and block loss artefacts were created. Values of thresholds for following metrics were settled: exposure time distortion, noise, slicing, freezing, and block loss. The paper: “Recent developments in visual quality monitoring by key performance indicators” was written. The video database at the PBZ server was cleaned up.
Some application and scripts for network measurements were implemented. Tests revealed high instability of packets sending and receiving time and due to this, the algorithm could not work. The research focused on other methods of available bandwidth estimation, which require streaming server modification and new features (like RTCP protocol) implementation. The changes or new implementation will also be required at client side. Another research results show that there is a possibility of using VTP (Video Transport Protocol) algorithm for video streaming. Analysis of packets streaming via Wi-Fi interface shown significant instability of packets sending and receiving times. The time differences were high enough to make impossible to implement the bandwidth estimation algorithms, which were based on fixed timestamps between packets. First tests of Extended RTP headers shown the possibility of using jRTPlib for this purpose and receive stream via standard video player, like VLC. A LibPacketAnalysis library was implemented which can be linked to any software and show the estimated bandwidth results. The code is in a development stage and has to be cleaned and documented. Using tc on output interface can be also a method for adaptive video streaming. When the player is MPlayer, there is a possibility to limit the bandwidth for particular stream instead of changing coding parameters. MPlayer displays the received whole frames like there was frame rate change.
As the project’s objective was to develop statistical models that will be able to estimate the quality of a video with regards to its usefulness in discerning visual information, for this purpose, some data must be gathered, which includes results of automatic quality assessment and recognition rates from experiments involving humans. This part of the project concentrates on performing calculations with objective measures and writing the results to a database.
A massive database has been created, storing several parameters coming from two different experiments: recognition of license plates, vehicle maker and vehicle colour in a parking lot, and recognition of different objects in different scenarios. Both experiments have been converted into a common format, as far as possible. In total, there are:
126 SRCs -> original video sequences
40 HRCs -> sequences modified (cropping, compression...)
1860 PVS -> derived clips
193 subjects answered in all experiments
69236 answers (data: rows in the database
This database is available in order to get parameters from there (such as bitrate, resolution, ...) and also to update it with other parameters (which need to be calculated).
Apart from that, all this data has been converted into a single numerical matrix in MATLAB, which will be the base for starting the modelling process.
Quality evaluation must focus on the areas where essential information is found. Some work has been done on following selected objects. It can be achieved in quite a simple way when standard foreground determination is possible. However, the method should also enable tracking in case of unstable background, and the objects may be nested in larger ones. For this reason, another approach tried was the use of optical flow. The implementation of this method found in MATLAB did not ensure sufficient accuracy. A more sophisticated algorithm is needed to reliably accomplish the task. Probably the best way to overcome the problem is to use a function from the OpenCV library.
There was also some attention paid to additional tools, like Kalman filter, which will be used to improve the accuracy and correct random errors. Methods of keeping track of multiple interacting objects were also analysed in case they are needed in the future.
To connect MATLAB code to database, external Java driver had to be installed and some configuration done. Available data about target recognition results is heterogeneous. Its structure is not certain at the moment, so the code for accessing it could not be completed. However, some of the design questions have already been resolved.
All the data available since the realization of the experiments (single values), i.e. answers from the viewers for every PVS (SRC-HRC), bitrate, resolution, scenario, etc. has been joined in a MATLAB structure to the results of having run the NR and FR metrics (single value for every frame -> arrays for every PVS). Three different sub-experiments (not in laboratory, at AGH University and by not practitioners respectively) from the objects recognition subjective experiment have been added to the two original sub-experiment (both in laboratory) included in the data. Taking into account the parking lot experiment, there are a total of 6 different experiments. Thereby, in just one MATLAB variable all the data required for the modelling process is accessible. This structure has a total of 69236 sub-structures with 23 fields each one: SN, SRC, HRC, recognized object, original object, bitrate, colour, make, file size, Levenshtein distance, viewer, resolution, scenario, correct, experiment, incomplete, blocking, blur, exposure, spatial activity, temporal activity, SSIM and VIF.
Analysing the likelihood of correct recognition from the 6 different experiments, the first decision taken has been to start searching for a model just for the 3 experiments, which have similar probability of correct recognition (around 80%).
First calculations (means, medians, and quantiles for NR and FR results) and conversions (creating arrays storing the answers for every PVS instead of doing it for every viewer) have been done to make easier to find the first model. Afterwards, just the mean will be used.
The first results showed us that there is no a clear correlation between any of the metrics used and the probability of correct recognition. This is the first conclusion exposed: better unique metrics should be used to obtain direct results.
Trying to create a generalized linear model was the next step. For this task, it was necessary to decide which set of videos was going to form the training set and which one the test set. Some linear models, starting from the most basic one, were tried. However, even the results from the most complicated of the models were not good at all. This model was taking into account every parameter, the square number of every parameter, and the total multiplication of all of them (both single and square).
As this model was totally unsuccessful, another approach was tried: neural networks. After running a program, which calculated, which neural network (set of parameters and number of hidden neurons used) was the one providing the best coefficient of determination R2, the best model found is the following.
Parameters used: Blur, Exposure, Spatial activity (SA) and Temporal activity (TA). Network structure, number of neurons in every layer: 7-8-1 (input layer – hidden layer – output layer respectively).
Algorithms dedicated to Classify Encrypted Traffic
In this work, we have proposed a payload-based method to identify application flows encrypted with the Secure Socket Layer/Transport Layer Security (SSL/TLS) protocol, which is a fundamental cryptographic protocol suite supporting secure communication over the Internet. Our approach consists of taking advantage of the information embedded in the SSL/TLS header to create statistical fingerprints of sessions to classify application traffic. We call a fingerprint any distinctive feature allowing identification of a given traffic class. In this work, a fingerprint corresponds to a first-order homogeneous Markov chain reflecting the dynamics of an SSL/TLS session. The Markov chain states model a sequence of SSL/TLS message types appearing in a single direction flow of a given application from a server to a client. We have studied the Markov chain fingerprints for twelve representative applications that make use of SSL/TLS: PayPal (an electronic service allowing online payments and money transfers), Twitter (an online social networking and microblogging service), Dropbox (a file hosting service), Gadu-Gadu (a popular Polish instant messenger), Mozilla (a part of Mozilla add-ons service responsible for verification of the software version), MBank and PKO (two popular European online banking services), Dziekanat (student online service), Poczta (student online mail service), Amazon S3 (a Simple Storage Service) and EC2 (an Elastic Compute Cloud), and Skype (a VoIP service). The resulting models exhibit a specific structure allowing to classify encrypted application flows by comparing its message sequences with fingerprints. They can also serve to reveal intrusions trying to exploit the SSL/TLS protocol by establishing abnormal communications with a server. Our analysis of the results reveals that obtaining application discrimination mainly comes from incorrect and diverse implementation practices, the misuse of the SSL/TLS protocol, various server configurations, and the application nature. Finally, even if we are able to identify some very reliable statistical fingerprints for selected applications, it is also possible to evade the classification by avoiding implementation mistakes and building the secure layer on limited, but widely used set of SSL/TLS states.
Reversible privacy protection with digital watermarking
The concept of reversible privacy protection is a new paradigm of selective content access protection. In traditional selective multimedia encryption techniques, the goal is to distort the whole image with the least possible computational effort. This is usually achieved by distorting just the visually significant portion of the data stream, i.e. the most significant bit-planes or low-frequency DCT coefficients. The proposed concept involves protecting access to selected media fragments only. This approach is motivated by the observation that in many cases the overall context of the presented content is still very informative and could potentially be useful for a more general audience.
The operation of the proposed reversible privacy protection uses a compressed reconstruction reference generated from and embedded in a digital image for the reconstruction of it’s original appearance. The detection and tracking module processes successive video frames and passes the locations of the privacy-sensitive regions to the privacy encoder using a binary protection mask. These ROIs are then blurred and the details necessary for prospective reconstruction are retained as a digital watermark in the remaining part of the image.The reconstruction is performed on demand on the client’s terminal only if valid access key is provided to the decoder.
The considered scheme operates on a discrete wavelet transform (DWT) image spectrum obtained using the biorthogonal CDF 9/7 wavelet. This results from the application of JPEG2000 for the compression of the stored frames.
In the first step encoder performs multi-level image decomposition. The coefficients that describe the sensitive regions and originate from all sub-bands from all decomposition levels are set to zero in the image spectrum. In the same time a certain portion of them is retained by means of a digital watermark. Generated watermark is then embedded using QIM algorithm in a way that each coefficient selected for information embedding carries a single bit of the watermark payload. In the last step the encoder synthesize image with watermark embedded in it.
The operation of the decoder starts with performing the same discrete wavelet transform as in the encoder. Taking advantage of the knowledge of the blurring procedure we can easily distinct protected and unprotected areas. Next, the decoder synchronize quantization step used in the embedding algorithm and extracts the watermark. In the last step, original content of the protected areas is recreated from the extracted watermark and the image is restored from obtained DWT spectrum.
For the proper encoder and decoder synchronisation and extraction only two parameters needs to be exchange between them i.e. protected image and the secret key. The rest of them is easily recoverable during extraction procedure.
Therefore the coefficient selection in the embedding step is determined in a pseudo random manner and depends on a secret key, it is required to posses a valid key to access protected parts of the image.
Visual description of this algorithm with some practical examples could be found at official INDECT YouTube channel: https://www.youtube.com/watch?v=IFEh8amSkpg
INDECT Prototypes
This section contains list of prototypes developed during the project. The prototypes were assigned to the following groups:
AUDIO EVENTS ANALYSIS
OBJECT DETECTION, RECOGNITION and TRACKING
CROWD BEHAVIOUR ANALYSIS
INDECT CRIMINAL ANALYSIS
SECURITY OF DISABLED PERSONS
IMAGE ANALYSIS FOR SECURITY PURPOSES
PROTECTION FROM DIGITAL THREATS
UNMANNED AERIAL VEHICLES (UAV)
INDECT INTEGRATION, DATA and PRIVACY PROTECTION
Group: AUDIO EVENTS ANALYSIS
Audio Events Detection, Classification and Localization
A module for real-time analysis of audio streams was presented. It performs detection, classification and localization of sound sources. It is trained to classify sounds related to threats (scream, explosion, gunshot, breaking glass). The information of localization is used to automatically position PTZ camera at the source and track it visually. The audio processing algorithms provide also a possibility of adjustable directional characteristic (overhearing a conversation from a user-defined direction in very noisy conditions).
Online Audio (Gunshot/Glass-breaking) Detection
Online audio detection demonstrated using graphical user interface of the detector running on one PC, and connected with audio cable with sound source, where different events are mixed with background noise with different SNR and sent to detector. The results contain also audio timestamp and event description.
Group: OBJECT DETECTION, RECOGNITION AND TRACKING
Visual Object Detection and Tracking - Event Detection and Automatic PTZ Camera Positioning
A module for real-time analysis of video streams was presented. It performs object detection, tracking, and classification. It is able to detect predefined events, such as: crossing a barrier in allowed/forbidden direction, various car parking events, abandoned luggage, person entering a road, and other. Detected event and object of interest are automatically tracked in a multicamera setup and available PTZ cameras can be automatically positioned to follow the object and provide high magnification image. Operation of the module was presented on live streams from outdoor scenes.
Surveillance and Localization of Detected Objects
The BUW node station is used for surveillance of an area of interest. Detected objects, e.g. pedestrians, will be located inside the world coordinate system, which can be expressed in GPS coordinates or in a kind of local world coordinate system using the metric “meters”. In other words, the local coordinate system includes a point in the field of view of the node station which has all coordinates equal to zero (center of the local coordinate system).
SSM station
SSM stations are designed as a highly modular system, that provides communication using different links. SSM are also equipped with sensors that can process signals received, examples include: plate recognition system or MiniDP devices. SSM stations are developed for obtaining data from tracking devices passing in its range. The station can be mounted at the entrances and exits of the underground structures, like underground garages, tunnels, subways or even at street crossings or junctions.
MiniDP tracking device
MiniDP device was developed to allow tracking of vehicles. To carry out the tracking function this device has been fitted with radio communication module working at ISM band, GPS receiver module, accelerometer and a microcontroller that integrates all of those elements and implements a communication protocol. All components were selected in terms of normalizing current consumption to provide the device with the longest battery life possible. By using the on-board accelerometer readouts microcontroller can determine whether monitored vehicle is not in motion and GPS module can be turned off in order to save power. One of the fundamental assumptions for the most effective design is to reduce the outline dimensions of the device to facilitate optimal placement in the tracked vehicle. With the use of neodymium magnets the process of placing MiniDP in a selected vehicle has been significantly simplified.
Vehicles Tracking Device
Tracking device was developed to allow tracking and positioning of vehicles and report through GPRS communication. The device is equipped with GPS positioning system, inertial positioning system, GPRS communication module, radio communication module working at ISM band, memory store and microcontroller. The microcontroller is responsible for the integration of the different subsystems integrated into the device and for establishment of the communications protocols.
Knife recognition
Recognition of objects resembling knives.
Mobile Object Tracking Module
The objective of Mobile Object Tracking module is to create a method which can track in a video sequence an arbitrary object pointed by a user. An implementation of this method will be used in Burzyk Unmanned Aerial Vehicle to follow any arbitrary target by the gimbal in such a way that the object will be in the camera’s field of view all the time. Therefore to cope with this task the real-time video processing algorithm running on the on-board embedded computer is the main requirement.
Group: CROWD BEHAVIOUR ANALYSIS
Crowd Monitoring
A module for crowd flow monitoring. It is based on processing of video signals from typical IP cameras (contrarily to dedicated Time of Flight cameras). The solution performs a real-time accurate detection and counting of people entering and exiting through a virtual gate.
Pedestrian Detection by Use of Boosting Algorithms Based on PTZ
Detection and tracking of pedestrians in the range of the PTZ node station, without limitation of moving or zooming the camera system (mobile camera). Detection algorithms are trained on examples of pedestrians and are not based on background subtraction. Combined with the localization feature, positions of detected pedestrians can be displayed inside a world coordinate system, e.g. a Google Maps image of an area.
People Detection and Tracking from a Top-View Position
This is a standalone system using Time-of-Flight (ToF) based camera images, captured from a Top-View position mounted perpendicular to the ground where people can reside. The camera measures distances (depth images) to each point in the real world. Filtering techniques are used to differ individual persons from a dense crowd of people in those depth images. Their trajectories are tracked over time and utilized for movement analysis, comprising counting of people, surveillance of hallways and detection of accumulations.
Embedded System for People Counting
The demonstrator presents implementation of the algorithm for detecting and counting people, who are passing under a camera positioned vertically down at the entrance to a monitored area (e.g. a building). The prepared algorithm takes the direction of the transition of a person into account. The model is realized as an standalone system i.e. on the Raspberry Pi microcomputer based on ARM embedded processor.
Crowd Motion Analysis
It presents intelligent video crowd movement analysis. The software collect information not only about the trajectory of movements of person, but also information about the frequency of selecting the routes, occupancy of particular areas of analyzed space and determining the areas and time periods. The system computes people density maps that allow to determine places where there is the greatest density of people or identify the occupancy of exits (including emergency exits) and escape routes without the need for laborious viewing of hours of video monitoring.
Density Measurements for Crowds based on PMD
Depth images are used to calculate the densities among crowds of people. With the projection of depths to a point cloud based 3D voxel grid, the occupancy is calculated and categorized to the resulting risk for persons.
Group: INDECT CRIMINAL ANALYSIS
The Lawful Wiretap Warrant System
A short video will illustrate the Lawful Wiretap Warrant system concept developed in INDECT and an example of hardware-supported content filtering and analysis.
System Babicka
System Babicka can search the important objects in text and their relation and subsequently to represent the relation by using graph principle. The important objects are nouns and defined another words. The Babicka system is implemented only for Czech language.
INSTREET System
INSTREET allows to pinpoint a location where a photo was shot by comparing architectural details of the example with its reference database. Just like textual text engine – it provides a list of results the user free to browse through. It was tested on an areas as large as 135 km^2. The result (median) is 0.05%, regardless of the size of the database. It means, that in median you will have to browse through 0.05% of your database to find your location.
Criminal Complex Analysis Platform
LINK is a complex analysis platform with rich features concerning data import. Unification and transformation. Mammoth is a tool for pattern discovery analysis. However it lacks advanced features concerning data loading. The integration allows Mammoth users to fetch data imported in LINK application. The integration is done using a shared-database approach.
INACT System
INACT is a software suite designed for law enforcement, experts, non-governmental organizations and all other entities involved in combating of Child Pornography. INACT software is divided into a set of two applications and a database module utilizing the Query by Example (QbE) approach.
FERRET – System for Searching Textual Data
FERRET is a system designed for searching textual data and social media. It allows creation of machine learnt patterns to be created and stored as profiles. The system permits real-time matching of new tweets against stored profiles to generate alerts. The profiles are learnt automatically from tweets and webpages that are marked as interesting by the user. FERRET employs SVM as the learning engine with a novel algorithm for feature selection and query expansion that achieves high precision and high recall.
CrimNet Tool
The CrimNet Tool is a web-based application to provide support for the process of managing information about organized crime gangs and monitoring their activity. The tool enables security forces to detect occurring threats and concentrate the resources on satisfying current needs.
AbcLog Application
Abclog is a desktop application supporting analysis of the log files. The main functionality includes automated retrieval of the IP information from the Internet services such as whois servers. In version 0.3 of the application operates WhoisIp (www.who.is) DnsStuffCom (www.dnsstuff.com) and RipeNet (www.ripe.net) services. It handles logs in Common Log Format, Combined Log Format, csv, xls and xlsx formats. AbcLog works on Windows (XP, NT, 7, 8) operating systems with Internet connection.
Indexing System – WWW Index
WWW Index is an indexing system for World Wide Web resources. It crawls data from web resources, indexes them, and stores the Web page content in databases. Furthermore, it can be used to offer various services based on the data stored in databases, like advanced searching and changes detection. It works properly for Polish web resources.
Digital Forensic Tool – PyFlag/FLUSH
PyFlag/FLUSH tool is an extension to well-known PyFlag digital forensic tool for discovery of application logs in the mounted file systems. The goal of the extension is to provide functionality of discovering images that are similar to the provided example. This can be helpful in detecting illegal material related to child pornography. The extension utilizes INACT for images comparison.
TouchGUI for Criminal Analysis
TouchGUI is a prototype of the touch-enabled desktop for criminal analysis. The realized demonstrator shows how such desktop can be constructed (widget-based concept allowing for extensibility and flexible when size of the workspace is concerned), what features it should include (graph, timeline and details visualizations) and how it can be operated using fingers (drag-and-drop, handling widgets, visualization control).
Blog Analyser
The developed system is designed for ormalizi the blogosphere, taking one of the largest polish blog portals – salon24.pl as an example. The analysis is based on the identification of groups of strongly interacting users. The formalism of users may have different characteristic features, so they can play different roles in communities they belong to. The system allows to identify significant roles played by users. Blogosphere is strongly dynamic so the analysis takes into consideration both evolution of groups and change of roles played by given users.
Group: SECURITY OF DISABLED PERSONS
Recognition of Traffic Lights
Traffic lights recognition in video sequences. The solution allows to detect traffic lights in real time. Software color filter is used to find candidates for traffic lights. Those candidates are then classified with machine learning algorithms and false alarms removed.
PC-based Blind Support System
Linux-based blind people assistance system supports people with decreased visual abilities. System uses eyeglasses with build in smart camera, intelligent controller with sensors and linux-based computational unit to understand scenery in front of individual. It provides information in form of sound messages about traffic lights, arriving city buses line numbers favourite places and different obstacles on the way.
Android-based Blind Support System
Android App supports blind people in their everyday duties.
Group: IMAGE ANALYSIS FOR SECURITY PURPOSES
Eye Localization in Face Images
Eye detection algorithm is a useful tool for normalizing and aligning face images. The position of eyes is found with use of neural network regression.
Automatic Plate Recognition System
The plate recognition system consists of system nodes placed around the city. The aim of each is to monitor the traffic and recognize every vehicle’s license plate. The nodes communicate over the narrow band wireless network. They can communicate with each other and pass the messages within network.
Panorama Visualizer for Fast Video Search
The demonstrator presents panorama based visualization and fast event search method for PTZ camera video. It generates a cylindrical (panorama) scene model and creates the video sequences containing selected panorama parts for a fast video search and convenient automatic and/or manual video analysis.
Stream Optimization
A stream for a camera is compressed by changing frame rate, resolution and compression. The obtained stream can fit a link capacity available in the network. Moreover, for given bandwidth the system is able to generate perfect stream by dropping some frames.
Face Recognition Module
The Face Recognition algorithms were designed to enrich the SSM2/SSO2 station. The main discussed scenario is picking some individuals from the crowd. They can be for example kind of criminals or people unwanted in some particular area. The main problem during the design was minimization of the false-positive detections. When system is aimed to pick somebody from the crowd it has to deal with big number of examples from negative class, while the example from positive class appears very rarely. One can see, that the system should have very high accuracy not to produce false alarms every five minutes.
Group: PROTECTION FROM DIGITAL THREATS
Intelligent Integrated Agent-based System
INDECT-WP3 Portal is an information portal providing a summary (including multimedia content) on all products realized within INDECT-WP3 work group.
A Platform for Analysis of Malware
Group: UNMANNED AERIAL VEHICLES (UAV)
UAV Burzyk
Burzyk UAV can be divided into small parts and transported in a backpack or a vehicle trunk. The longest element is 80 cm long and the assembly time takes up to 10 minutes. Thanks to its compact size Burzyk can be launched from hand, catapult or specially designed cart, and the landing is performed on its chassis. It is equipped with an optical or thermal gimbal and it sends live video straight to the ground station. It is also equipped with onboard computers. The most important ones are mission computer and image processing computer.
UAV Ground Station
UAV Ground station is responsible for running the UAVs’ applications and managing the UAVs’ missions. Ground station consists of:
Rugged computers with installed software for UAV operator.
Communication mast
Visual landing system hardware
UAV Autopilot
The primary function of an autopilot component is to control elevator, rudder, ailerons deflection and throttle setting in order to autonomously steer the aircraft. Such a functionality requires direct actuators control and the autopilot becomes a crucial component required to perform the flight. In current autopilot design this control can be switched to common RC radio control at the electronic level. Thus an autopilot module also includes the possibility to allow the human-controlled flight when necessary.
Return – Application Server
It is a central point managing the flow of communication in WP2 subsystems. Return is a modular approach for developing server applications. A core return system is responsible for loading, running and managing running modules that provide functionalities. It also provides methods for remote clients to connect to the server, authenticate and be authorized to access modules. Every module can provide functions accessible from other modules or from remote clients. What, when and how is available depends on the module and access configuration include in Return.
UAV Mission Application
The mission subsystem corresponds to the deliberate layer in the three-layer architecture for robotics software. Its purpose is to:
facilitate accepting high-level description of UAV flight goals
convert these goals into simple actions that can be understood and executed by the executive layer (i.e. guidance)
and request execution of these actions in the appropriate order so that the flight goals are fulfilled in a potentially unsupervised manner.
Group: INDECT INTEGRATION, DATA and PRIVACY PROTECTION
Integrated Access to Services
The Integration System (IS) enables access to services to end-user. Integration includes the following layers: physical (data source) level, data processing level, high-level service (analysis, decision making). To achieve scalability, the system for integration does not run any tasks other than tasks for integration, logging of client requests and end-user authentication. The IS has no direct access to data sources, and it calls remote services. The IS has functionality of User Interface. End users can define a set of functions offered by the UI.
Prototype of INDECT Portal Document Management Subsystem
INDECT Portal Document Management Subsystem (DMS) is a Web application that supports police officers reacting to crisis situations in their online-represented decision making. The system coordinates the workflow of actions performed by the involved officers, the access to related documents and remote INDECT services in accordance to predefined procedures. The system is designed in a manner that allows to control how it is being used by portal operators what significantly reduces the likeliness of the system’s misuse.
Integration Platform
The Integration Platform is a command and control system which interacts with different Indect applications. The basic functions are event notification and surveillance for operators. The notification is about events, which are detected by Indect WP7 algorithms. Surveillance is done by audio or video streams delivered by node stations of WP1
Node Station and Central Station Framework
A modular system for data acquisition, processing, and transmission. It allows to create a data processing pipeline adapted to user’s needs. It is used as an audio/video/and normalize signals processing framework, both by developers and by end-users. The framework is platform independent (Windows, Linux, Mac, smartphones) and algorithms are developed to be mutually interconnected and exchange data in parallel. Examples of parking lot area monitoring, outdoor monitoring, bank area monitoring and other were presented.
IVAS – Interactive Video Audio Service
System IVAS is communication system which allows managing the communication for defined area of participants (end users). The central manager with web application can manage all text, audio and video communication for all end users. It means
• To manage the call between all end users. Manager can connect two end users, can establish conference call, and can send report to all or group of end user by using technology text to speech.
• To manage video stream from end user to center or opposite
• To manage video file form end user to center or opposite
• To manage text information from end user to center
Node Station based on ADTF integration Platform Communication
The Communication between a central integration platform, developed by PSI, and the BUW node station is based on an ADTF framework. For data acquisition and image processing, algorithms can be integrated into the framework by self developed dynamic linked libraries. An integration platform user is able to interact bidirectionally with the BUW node station. Different kind of events can be requested, streaming of videos can be controlled and a certain set of parameters (pan, tilt, zoom, kind of report) can be changed.
Social Threat Monitor
Social Threat Monitor (STM) is a novel Web system for documentation of user-defined threats. It contains a Relational Database and user interface based on a GIS component. The description of threats can be provided by all registered users, so the data acquisition is performed in a distributed manner through Internet. Threats are presented on a map. Combined relational and related to area search and filtering of threats is implemented.
Bluetooth Authorization and Detection
A module dedicated to detection and identification of Bluetooth devices. It allows for recognition of authorized users, and for proximity assessment. In presented scenarios loss of a signal indicates theft or abduction of a protected goods/person.
Detection of Secret Message in Static Images by Universal Steganalysis
An image steganalysis module for detection of secret message was presented. It is based on extraction of statistical features from image database (includes cover and stego images in JPEG image file format) and it also includes creation of model using classifier. Subsequently, trained model allows detection of secret message that was inserted using popular image steganographic methods.
Malicious Image Tampering Identification System
Malicious image tampering identification system (MITIS) is a state-of-the-art solution for active protection of digital images against forgeries. It allows for accurate identification of image regions affected by unauthorized content modifications. One of the most important features is the capability to restore the original content of the tampered fragments. The reconstruction procedure is highly reconfigurable, and features several quality levels. The tampered fragments can be restored when even up to 67% of the image area becomes tampered. When implemented directly on the acquisition device, provides efficient, state-of-the-art protection for digital evidence.
INDECT Block Cipher
One of the main goal of WP8 work was development of cryptographic solutions to ensure data confidentiality and privacy. During the exposition, the crucial achievements will be presented – software and hardware implementations of cryptographic algorithms. Mainly, the practical applications based on a new symmetric cipher – INDECT Block Cipher (IBC). Also, other techniques such as stream ciphers or hash function will be shown. Additionally, some other solutions (i.e. error detection methods) which increase the security will be presented.
Potential Impact:
INDECT project was performed in a close cooperation with End - Users, mainly Law Enforcement Authorities from different European countries. The cooperation enabled for efficient practical impact of research done in the project.
End Users Requirements Board (EURB) was established as an advisory body of the project. Representatives of the following entities were engaged in the EURB activities:
Police Service of Northern Ireland
Polish Police
Czech Police
Guardia Civil (Spain)
Police of Latvia
Police of Malta
KEMEA - Ministry of Public Order and Citizen Protection in Greece
Polish Platform for Homeland Security
Romanian Police
A number of INDECT solutions were successfully tested by Police forces. An example is the application for combating Child Pornography developed within the INDECT Project was tested by the INDECT partner - GHP. The interest for testing this application was expressed by the Czech Police (which was not part of the INDECT consortium). An agreement for testing of the INACT application between INDECT Coordinator and Czech Police was prepared. The application was delivered and installed for Czech Police for testing.
Another exploitation of the project results was a project dedicated to design an application and an intelligent system supporting people with sight disabilities.The new project benefits from basic methods developed by INDECT like image recognition (e.g. bus numbers) and sound recognition (speaker’s announcements) which are most applicable for people with sight disabilities and the blind.
The know-how from the INDECT project is going to facilitate the final product development. The techniques of project coordination and deployment will improve the new project cycle.
As a follow-up of INDECT as the research project, it is planned to set-up a spin-off company in order to commercialize prospective results of the project. For example, al support from the AGH was announced as concerns establishing a spin-off company in collaboration with the AGH’s Academic Incubator of Enterprise in order to set up the new venture.
The licensing will be commercial with some potential licensing at no charge for research and the University is taking the leading role as the INDECT project coordinator.
Solutions elaborated in INDECT constitute a foundation for creating new tools and prototypes that enhance security of citizens. These technologies, including automatic threat detection, watermarking and privacy and data protection, can be reused by companies in this domain.
The theoretical knowledge and lessons learnt from the project are shared with the students and are the valuable input to the content courses and seminars. Additionally, the students get the unique possibility to actively participate in test scenarios and have chance to validate the theoretical knowledge in practice. The research staff participating in the project benefit from the exchange of knowledge with other European partners which also positively impacts on other research projects. It should be noted that the networking build on this project leverages further across European collaboration in new research initiatives. The theoretical results of the project are being disseminated to the scientific community at the special conferences and events organized by INDECT partners.
Societal Impact of INDECT
Analysis of Societal Impact of the project was performed by an external expert in this area, Peter Burges, Research Professor at the Peace Research Institute Oslo (PRIO) and Senior Researcher at the Institute for European Studies at the Vrije Universiteit Brussel. He is Editor of Security Dialogue, an international scholarly journal dedicated to innovation in security research and Series Editor of the Routledge/PRIO New Security Studies collection.
Societal security needs addressed by the research
The INDECT project responded to the FP7 ESRP call for 'intelligent urban environment observation system' (SEC-2007-1.201). The project summary sets out its primary technological objectives, namely:
to develop an intelligent system for automatic detection of threats and recognition of criminal behavior or violence,
to develop new methods and techniques providing tools to support activities of Police officers, including tools for threat detection in the internet; this includes the development a new type of search engine combining direct search of images and video based on watermarked contents and storage of metadata in the form of digital watermarks, and
to develop techniques for data and privacy protection in storage and transmission of data based on quantum cryptography and new methods of digital watermarking (INDECT summary).
These primary aims were based on a number of assessments and assumptions about the needs of society. The project takes its point of departure in a certain set of assumptions about the security of citizens and what it describes as the 'weak point' or vulnerability of European critical infrastructure. This vulnerability is European critical infrastructure and of
Security is becoming a weak point of energy and communications infrastructures, commercial stores, conferences centers, airports and sites with high person traffic in general. Practically any crowded place is vulnerable, and the risks should be controlled and minimized as much as possible. Controlling access and being able to respond rapidly to potential dangers are facilities which every security system for such environment should have. One of the best ways to tackle such problems is through active observation. This explains the real need for intelligent monitoring and information systems in urban (and wider) areas (INDECT summary).
The project description asserts that security is 'becoming a weak point', but the nature, causes, meaning and implications of this 'weak point' are not described or in any way documented by the project. This is a particularly critical point and would give considerable force and legitimacy to the activities of the project. If this 'weak point' were documented, described and analyzed as a social phenomenon, the project's relevance would be anchored. While in a very broad way the assumption of a series of security weak points might well be thought to be reasonable, they in actuality represent a vast range of complex social, human, political, cultural and technical relations. Assuring both the project's technical success and its political support in terms of democratic accountability implies clarifying and following up on the societal meaning of these premises.
The vulnerability of 'energy and communications infrastructures' to either malicious attack or catastrophic failure is largely a technical one. These technical systems can be damaged or destroyed through mechanical causes, either through intentional sabotage or by one form of accident or another. There is a general belief that dependency of European society on such technical infrastructure is increasing. This belief, which is not detailed or documented in the project, is linked to implicit and explicit understandings of societal security and how these strengthen or weaken European dependence on critical infrastructure. Such dependence varies considerably according to local society, cultural, tradition, family, living style, economic wealth, and geographical variation.
A more thorough analysis of the societal security of critical infrastructure would also focus on the fact that terrorist attacks as a rule are not organized instrumentally in such a way as to realize the goal of interrupting the resources delivered by critical infrastructures. Terrorism is carries out symbolic acts, organizing events that create meaning, reminders of vulnerability. The aim of terrorism is not to stop the flow of electricity, to take one illustration related to critical infrastructure, but rather to create fear and unease. Addressing terrorist threat to critical infrastructure by focusing on purely technical function of such infrastructure risk missing the point of terrorism and thereby failing to protect against it.
The security of 'commercial stores, conference centres, airports and sites with high person traffic' poses a different set of challenges related to societal security. Some of these are linked to the questions of privacy and data protection that are evoked at several moments in the INDECT project description and scrutinized in detail by the 'Ethical issues' reports (D0.5 D0.6 D0.7). Other central concerns for societal security are not covered by the primary aims of the INDECT project, concerns that, if carefully accounted for, would strengthen considerable the impact of the project on the actual security of citizens, and on public perception of the value- added, relevance and accountability of the project. First and foremost, the project's basic premises do not contain a consideration of the social and cultural dimensions of public spaces in which the systems and measures developed and proposed by INDECT will deployed and expected to function and bring security results. Public spaces different considerably in their social organization, and thus in the way that dangers are perceived and played out them. A shopping mall, a railway station and a public square, for example, differ greatly in terms of both the objective safety of those who circulate their and also in terms of the ambiance of security and vulnerability, that different urban spaces create and afford. There is, in addition, numerous nuances and distinctions between such public spaces in different cultural, social and geopolitical situations. The difference between small cities and large, southern and northern cities, coastal or plane areas, rich and poor, arid or humid, etc. Furthermore the security needs and demands of citizens in general vary greatly from Member state to Member state, from region, from one economic class to another, according to language, family, employment, etc.
In some cases more technological security measures of the kind proposed by INDECT enhance security. In other settings, more technological security actually lowers security, causes insecurity, etc. While it is true that many crowded places are vulnerable, it is not always for being crowded that they are vulnerable. Nor is it clear to what degree this insecurity stems from the crowdedness or not. In any case, the 'risks' to which the INDECT description refers are in all likelihood highly complex and socially differentiated as described above. The assumption of a 'need' to reduce such risk is neither substantiated by social risk research, nor politically justified, even if such a justification and legitimization would be easily obtained. The public acceptance of security measures that border on interventions into public space or the private sphere would likely be much stronger if these questions of social compatibility were dealt with and publicized accordingly.
Similarly, 'controlling access', will have both societal costs and benefits when it comes to the security of citizens. It will be a viable tool for the protection of citizens against threats and dangers only under specific, socially determined conditions, that is, as a function of how movement and flow is structured and how it flows beside, though, and around potential dangers. The value of access will be display differing characterizes in differing settings, and the social traditions and institutions that assure the flow of citizens in and through liberal market models.
By the same token 'potential dangers', such as they are described in the INDECT project, are only partially objective, concretely measurable, identifiable and traceable as dangers or threats that can be simply removed from society and dealt with as foreign agents. The perception of danger-like the social acceptance of the security measures designed to address danger--is socially complex, woven into understandings and experiences of technologies, security actors and institutions, and they way their presence and actions are woven into the dangers themselves.
The basic tenet of societal impact review is that security is a societal challenge. The ultimate object of all security measures is people. The primary question for societal review thus becomes: how does the reflect this awareness in its principles and procedures. While any given security challenge can most certainly be addressed by technical means, it can only be said to have met its aims if it makes people more security in meaningful and verifiable way. A detailed examination of these issues in the development phase of the project would likely have the double of effect of the making the project considerably more effect-since it would in that case actually be addressing scientifically confirmed security problems.
Approach to societal needs: From security measures to security threats and back
The security weak points set out in the basic documents for the project are ultimately societal. They have clear technical dimensions (concerning the robustness of technical systems, communication procedures, coordination, harmonization of technical equipment, etc.) Yet these technical dimensions are embedded in societal contexts, reflecting societal needs, and positioning possible solutions squarely in options provided by societal limitations, be they public fears, cultural customs and traditions, ethical constraints, legal regulations.
The security weak points identified by the project are not by any means incorrect or unreal. They are plausible from a technical point of view. The challenge lies in the fact that they are also deeply social, implying important assumptions, and considerable consequences. If these weak points are left in the discourse of technology, natural science and the hard logics that accompany them, they will themselves inevitably be fused with threat itself. Security research that does not take the consequences of itself being socially embedded with deep social linkages at all levels, will not be able to respond to insecurity, which itself is fundamentally social.
Moreover, an approach to security that revolves around technology for technology's sake not only risks not adequately addressing the insecurity of society, it also risks generating new insecurities. The old geopolitical version of the 'security dilemma' noted that nation states that arm themselves against belligerent neighbours do little more than make those neighbours more insecure, thus obliging them to upgrade their armaments, thereby increasing the overall insecurity. A parallel effect is generated by advancing security measures that do not communicate with the needs of those in society whose security is in question. The new security dilemma of our time: Blindly technological security measures have a tendency to generate new security threats as a secondary effect of protecting against the principal threats.
Threats to society addressed by the research
The INDECT project proposed technological measures and strategies for the development of technological systems to address threats of violence to urban areas, in particular public spaces, threats to internet infrastructure, tracking of objects and individuals, new technologies for digital watermarking and tracking those involved in child pornography and trafficking in human organs, tracking of individuals and transactions involved in various types of fraud. It also aims to develop internet-based tools for preventing crime and terrorism. It will develop high capacity, high speed computer tools for processing data about potential and actual crimes. In addition, it aims to create tools and technology for privacy and data protection during storage and transmission of information.
In short, the INDECT project aims to address a variety of perceived threats to society: violent, criminality, terrorism, particularly in urban settings, child pornography, trafficking and fraud. On a very general level, these threats have currency and are more or less uncontroversial. These are threats to the security of society. They interwoven with society in complex ways. Most prominently, the threats themselves are not objective, but rather are products of perceptions and values, from cultures of fear and safety, from experiences and understanding of how various dangers function in society, how they are dealt, in what ways they are increased or decreased. Criminal violence and terrorism are most certainly threats to the security of society, and yet understanding how to reduce threat requires understanding society's experience of that threat. The key to assessing the project's overall societal impact begins with the question of how the threats it aims to address are linked to societal concerns, i.e. its values, customs, traditions, insecurities, robustness, resilience, etc.
The project's aim of generating technological solutions to track objects and individuals, through advanced surveillance tools, watermarking, and technologies of identification and tracking. intend to address threats. But given that they address potential or future threats, society's acceptance of security measures and insecurity toward future threats will be clear elements in the analysis. The guiding in principle of all such pre-emptive crime-fighting is that the crimes lie in the future, that their reality is a potential, that they are something to come. Crimes and acts of violence of the kind to be handled in the INDECT project are understood through assumptions and informal processes of speculation. Unlike preventive research on hazard or crisis management INDECT is anticipatory. It targets actions that have not happened yet. This ground for this speculation of threat requires a deep understanding of society and its security needs.
Approach to threats
Not only the degree to which the fear of such threats is generated in and through society, but also the basis for developing technologies aimed at addressing such The primary principle when seeking to understand the approach to threats in terms of societal impact is that security measures have a societal correlate. Their success or failure depends to a varying degree to the character and values of the society in which they operate, and on the degree to which the measures are tailored to and respect the complexities of security in society.
Beneficiaries of the research
It is simple to imagine and in general taken for granted that the primary beneficiaries of the INDECT project are European citizens. However, it is important to examine closely the project's means and aims, and reflect over the complex relations that these have to citizens.
Much of the ethical justification for the project's aims is drawn from a relatively uncritical assumption that good will and instrumentally sound research design will bring favourable research results. This is a demonstrably inconsistent assumption. Moreover, the more a project is complex and, layered, and long, the more the path between its good intentions and robust methods that lie at its outset and actual outcomes becomes invisible for those with either curiosity or concern. This invisibility in turn generates insecurity that may interfere with or even subvert the project's very ability to contribute to security.
This additionally due to the fact that the societal world view of the experts and specialists that have designed and carry out the project is likely quite different from that of the citizens that are intended to benefit. Neither is right or wrong. Elements of both will need to be brought into the project's overall perspective in order to make valid for all. This is above all crucial when seeking to understand and then improve public understanding of and attitudes toward the project’s means and aims.
All security measures, be they in the form of technologies are human actions, tend to have secondary effects that can often conflict or come in contradiction with the aim of security. Those that are not reflected, legitimated and justified in social and cultural terms, will further detract from its ability to bring security benefits.
Privacy, to take a central example from the project, is not only a right. It is also a fundamental human need. The self, the personal sphere, the home, the family, etc. have value unto themselves. For some, they are sacrosanct. Compromising privacy, as so many of the new technologies tend to do, will weaken the benefits of privacy in way that cannot be reduced to a 'balance' between security and privacy.
An important part of the challenge involved in the work of INDECT is that the project's scope and complexity is uncommonly large. The vision of security it builds upon is broad and comprehensive, involving many actors and many interested parties. The perspectives behind the technologies it proposes to develop and implement, and links and logics that join them are far from obvious, even for internal participants of the project. This has a distinct effect citizens' reaction to the work and, thereby, to the potential security benefits it will potentially bring. The path that joins the means and ends of the project is largely invisible. And yet it is precisely this path, and not the advanced technological accomplishments alone, that will bring the benefits of security to society.
The sophisticated tools under development for the analysis of video and audio data, for example, may be envisaged to benefit citizens and groups in the long-run or on the large scale. However, from the perspectives and experiences of everyday citizens they may be experienced as intrusion, because the larger or longer term benefits exceed the perspective. It is important to cast light on the differentiation of benefits along different levels and different time-frames of the project.
By the same token, computer aided detection systems, electronic watermarking, or quantum cryptography may only provide security within a particular horizon of possible crimes or actions. Most citizens are not concerned by what happens within this picture. Even though the security measures are taken in social settings, and even though the threats they seek to address are also imbedded in society, the complexity of society and the differentiate perceptions of threat, understandings and experiences of threat, and exposure or vulnerability to threat No project can solve all security problems for all people. An awareness of what the INDECT project cannot do, and an understanding of the false securities and insecurities it can generate as side effects in society is fundamental. This is the case both internally in the framework of the project and externally in the public sphere.
Security research does not benefit everyone in society on an equal basis. Some do not benefit at all. What are the correlates of those that benefit from security measurements? What distinguishes them? : economic well-being, social standing, class, caste, family or social background, education, etc. Such properties and be simply correlated with and the relates of prejudicial benevolence.
Society as a whole as beneficiary
In this sense it must be emphasized that benefiting society as a whole is a difficult challenge. And yet it is essential to ask the overall question of societal benefit. Framework research, funded by public monies and with a considerable demand of public accountability. Society at large counts and should be considered in the justification of measures and of scaling of measures.
Clearly, INDECT will benefit society unevenly. No research project can meet the needs of society in its entirety. However, the particular inequalities of distribution of the work on INDECT should be explored and explained, including the considerable benefits that will flow to the industrial private sector. European society can, in general terms, be described. Its priorities, values, needs, and requirements can indeed be mapped and the results fed to both those who engineer the technologies at the heart of INDECT and the dissemination and communication challenges that will follow it. The overall societal benefit of the project must be explained and understood on a higher, political level. Like any research project within the European Security Research Programme, INDECT should be exemplary. It should in what it undertakes to do and also what it chooses not to do, it should embody the political notion of what is the best for all of society. This description and explanation can be put in terms of societal values.
The point of entry for such an analysis is the question of the value the measure have. What are the values that intelligent analysis of video and audio data can preserve? How can one be clear about the general value to society of search engine designs supporting semantic searches based on a watermark system? How will intelligent processing of data serve society?
In all of these communication issues, the challenge boils down to transcending the technical vision of the research, making it sensible to the principles of the EU or any free society that such research will be deployed in.
Ethical Issues
The project faced several ethical issues related to its potential impact and use. Handling of collected personal data while protecting privacy and confidentiality is of major importance for the project; therefore a specific task monitors on this aspect throughout the project. For this purpose Ethics Board was established which supported the project consortium in examining the societal, political and legal aspects of potential applications (especially dual-use applications), defined and approved the future exploitation plans of the project results, and advised in dissemination and communication strategy of research results to a wider audience.
All of the research activities within INDECT project were carried out so as to ensure the appropriate balance between the protection of the rights of the individual and the protection of society. Ethics Board had for the main object to ensure strict compliance of research outcomes with already established rules concerning privacy, data protection, to ensure genuine informed consent of all those participating in the project, and to ensure that information is only used for its intended research purpose. It was also responsible for managing and monitoring all ethical aspects of the project. These aspects included the promotion of gender equality.
The Ethics Board has a broad membership, designed to exert strict control over the project. It included representatives from data protection experts, Non-Governmental Organisations, academic world, industry and the Police Service of Northern Ireland, which is generally recognised as adhering to the highest standards of human rights protection in all aspects of its work. The Board did not view its role as ensuring compliance as a minimalist task, solely designed to ensure legal compliance. Rather, it perceived its function as broader, including overseeing scientific and societal issues related to the research activities conducted within the project.
The algorithms and methodologies underlying the project relied primarily on previously available, usually public, information sources. These include monitoring cameras, public Web pages, etc. The research covered by the INDECT grant, did not consider processing of highly sensitive material, such as telephone intercept, VoIP, etc.
The main objective of INDECT research was to make the monitoring and search process (and procedures) more automatic. This will ultimately allow for more informed decision-making. The value that will be added by deployment of INDECT research outcomes is that existing systems would operate with less human intervention, which will lower the level of subjective assessment and the number of human mistakes. This means less staff will be required for supervision of surveillance activities (e.g. monitoring of CCTV camera networks). This will resulting in less opportunities for illegitimate use of such information, or for human error to result in violations of the rights of the individual. There will also be economic benefits, in terms of the reduced staffing requirements. Police officers could be freed up to carry out frontline policing tasks.
The algorithms that are under research are intended to be used solely in cases where this is justified. Existing legal requirements will apply where the police or other agencies seek to use tools based on INDECT research outcomes in any investigation. These tools will not, and could not, involve any reduction of existing international and national legal protections. Integrated safeguards, such as blurring, etc., allow sophisticated controls to be placed on images (e.g. car registration plates) within recorded images.
Tools based on INDECT research will enhance the ability of the police to protect the public. An example can be an algorithm for detections of dangerous situations, such as: abandoned luggage or withdrawing a dangerous object (a knife or a gun). Such tools will allow the police to target resources where they are needed. It would not involve mass surveillance, but rather targeted efforts towards detecting the real threats.
It is important to note that a person highlighted by tools based on INDECT detection algorithms would merely be brought to the attention of the relevant authorities, so that normal lawful measures can be taken. The fact of highlighting by itself would involve no legal consequences for the person, and no permanent record would be kept, unless there was a specific legal reason to do so.
Tools based on outcomes from INDECT research are going to be designed to protect the public and its use in any given situation will have to comply with national and international rules regarding privacy. In terms of its design and implementation, the EU Charter of Fundamental Rights applies. As a result of the Lisbon treaty, the EU became a party to the European Convention on Human Rights. These safeguards are additional to those already enjoyed throughout Europe.
National Standards regarding privacy (e.g. the Data Protection Act 1998, the Human Rights Act 1998 and the Regulation of Investigatory Powers Act 2000 in the United Kingdom) will also apply in every EU country.
The sentence: “if you have done nothing wrong, you have nothing to fear” is only true if every aspect of the criminal justice system works perfectly, on every occasion. Tools based on INDECT project research outcomes will provide EU Member States with the technology to ensure that decisions around public safety are based on the maximum amount of relevant information available.
List of Websites:
www.agh.edu.pl/en
Coordinator:
Prof. Andrzej Dziech
Tel: +48 (12) 6172616
Fax: +48 (12) 6345582
E-mail: dziech@kt.agh.edu.pl
AGH University of Science and Technology
Al. Mickiewicza 30
Poland – 30-059 – Krakow