Progress beyond the SotA and results
SURVANT aims to develop a product that will introduce solutions beyond the industry state-of-the-art to face the challenges identified in the markets targeted. A comprehensive list of those challenges, the solutions envisioned and the ambition involved, indicated by their technology readiness level (TRL), is provided:
1)Scalability: SURVANT will address system scalability issues that emerge from the explosion in the amount of available video content employing and extending the OpenZoo framework (TRL 7). OpenZoo is an open-source, MIT licensed, distributed, stream and batch processing framework and includes remote, automated deployment of processing services, transparent communication and allocation of resources, and cross-platform support. OpenZoo has been tested as a real time search and analytics framework, based on images shared through Twitter, during the CUbRIK project (GA 287704).
2) Video analysis: SURVANT will perform video analysis employing Deep Learning (DL) techniques (TRL 6). Specialized research will be performed to analyse the current state-of-the-art DL implementations and adjust them to the specific needs of surveillance video processing. Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) will be used to analyse static and motion content, respectively. Inter-camera tracking and re-identification will be at the core of attention. Optimal balance between speed and accuracy will be pursued.
3) Event Enrichment and Reasoning: SURVANT will deliver an inference framework able to combine together low-level information and semantic annotations to enable automated reasoning mechanisms to discover high-level events and/or investigative hypotheses. SURVANT will evolve the OWL tableau reasoning framework developed in ADVISE (TRL6), based on a SWRL (Semantic Web Rule Language) approach, applying the event calculus formalism in order to allow the event reconstruction in a narrative way taking into account spatial-temporal coordinates useful to track the crime and predict its evolution in the time and space.
4) Video and Image indexing: One of the main challenges when dealing with large scale image/video retrieval application is how to efficiently index the content extracted from large image/video repositories. While handling semantically-mapped content (event type and attributes) is relatively straightforward, using well-established data management systems, low and mid-level features are much more difficult to handle efficiently, due to their high-dimensional aspect.
5) Intelligent, human-in-the-loop, GIS-based user interface: In the scope of ADVISE, a GIS-based user interface (TRL 7) has been developed and demonstrated, allowing the user to execute targeted queries, in terms of location and time, and visualize the returned results in analytical or summarized forms. In LASIE, advanced relevant feedback tools (TRL 6) have been developed allowing the user targeted queries in terms of semantics. The user is assisted during the query formulation phase, while the query expansion is focussed on semantic analysis and takes into account user's interests by the analysis of previous queries.
6) Legal and ethical issues and LEP framework: SURVANT will deliver an intelligent and extensible framework for automated video analysis and analytics respectful of human rights according to European legal framework for privacy and data protection, avoiding disproportional use of personal data. ADVISE had adopted a privacy by design and privacy by default approach to develop a Content Agent Mediator (TRL7) focused on ontological structures in the specific field of surveillance to ensure privacy protection.
Based on the results already achieved, key functionalities have been identified that can provide high added value for potential customers compared to the competition have been identified and presented below:
1) Situational awareness framework.
2) Advanced content-based search.
3) Search expansion tools.
Potential Impacts
SURVANT will have a huge impact in terms of:
1) Improved productivity and effectiveness of LEA's crime analysts and operational teams;
2) Increased capability in crime prevention via a quicker and more effective criminal trail tracing and a quicker identification of crime behavioural patterns with the consequent reduction of the impact of criminals’ activities;
3) Improved capability of LEAs to analyse day by day the increasingly huge amount of surveillance systems videos and the reduction of the gaps between LEA’s capabilities and the growing variety and volume of 4) Develop a video analysis and event detection technology capable of discovering correlations among criminal facts and detecting criminal movements before and after the suspicions event, meeting the needs of European LEA and contributing to the fight against crime and terrorism;
5) Seamless interfacing with multiple heterogeneous repositories/sources to retrieve relevant videos and available metadata;
6) Exploit the partners’ innovation capacity to efficiently integrate the proposed innovative system into a pilot that will be set up at the ADM premises;
7) Strengthen the competitiveness and growth of the industrial partners, opening a complete new market.
