Multimedia Analysis and Correlation Engine for Organised Crime Prevention and Investigation

Project Information

MAGNETO

Grant agreement ID: 786629

DOI

10.3030/786629

Project closed

EC signature date 23 April 2018

Start date 1 May 2018

End date 30 April 2021

Funded under

• Secure societies - Protecting freedom and security of Europe and its citizens

Total cost

€ 5 320 475,00

EU contribution

€ 5 320 475,00

Coordinated by

EREVNITIKO PANEPISTIMIAKO INSTITOUTO SYSTIMATON EPIKOINONION KAI YPOLOGISTON
Greece

Periodic Reporting for period 2 - MAGNETO (Multimedia Analysis and Correlation Engine for Organised Crime Prevention and Investigation)

Reporting period: 2019-05-01 to 2021-04-30

Summary of the context and overall objectives of the project

Investigators all over Europe are very often confronted with a variety of complex cases in which huge amounts of heterogeneous data have to be analysed for the prevention, investigation and prosecution of criminal offences. The analysis of such data volumes frequently exceeds the capacity of the police teams. Moreover, in some cases, both the workload and time constraints can be extreme, such as in a sequence of terrorist attacks which, from the LEAs' part, involve pre-incident intelligence analysis, real-time incident processing and post-incident evolution prediction linked potentially to other threats or subsequent attacks. In this light, MAGNETO aims to enhance the capabilities of LEAs in managing, investigating, correlating and reasoning upon huge volumes of heterogeneous and disjoint multimedia data, in order to accomplish highly-efficient crime prevention and investigation. Specifically, MAGNETO provides powerful, intelligent, intuitive and modular analytics tools through which investigators can receive comprehensive answers to their queries and discover relations between data more easily, gaining better insights. The cornerstone of MAGNETO is, accordingly, the definition and specification of the common representational model for data, together with the development of advanced semantic reasoning and augmented intelligence tools to support LEA officers, integrated in a common platform that allows storage, handling and processing of the collected multi-source information.

MAGNETO's objectives are to:
- Augment the capabilities of Law Enforcement Agencies (LEAs) in managing, investigating, correlating and reasoning from huge volumes of heterogeneous and disjoint multimedia data.
- Develop dedicated technical breakthroughs to anticipate both the growth in volume and types of data, and the resources and evolution of crime and criminality.
- Design and compose an open framework and develop powerful tools to deliver increased innovation capacity in Europe, and more efficient information exchange between operators and other users.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The results of MAGNETO revolutionize the capacity of Law Enforcement Agencies (LEAs) to deal with extreme volumes and diversity of data in order to accomplish highly-efficient crime prevention and investigation. MAGNETO has developed over 30 different tools and modules integrated into a common Platform that is available through 4 different flavors (business products). These tools range from text mining, data mining, machine translation and video analysis tools to sophisticated information fusion and semantic reasoning engines, as well as visualization means. The MANGETO Platform is underpinned by a security ontology that serves as a common representational data model, fusing information coming from various sources and formats, and enabling further knowledge mining and information inference. Technologies related to maintaining the chain of custody were also further developed (PRO-V ontology, immutability using IPFS and Blockchain).

MAGNETO has carried out a series of 12 feature-rich demonstrations to internal and external LEAs, policy makers, external advisors, and other key stakeholders. Dedicated training sessions were organized and appropriate training materials (structured into 8 different modules) were produced, suitable for online training (in response to COVID-19 challenges). On-premise installation and deployment of the MAGNETO Platform was achieved in 7 different countries.

Legal and ethical guidelines were transformed into appropriate impact assessment tools as well as enabled relevant technical design choices. Outreach activities were intensified through the use of appropriate dissemination and communication measures. Three dedicated task forces were set up to perform market analysis and positioning, clarify IPR issues and formalize the IPR framework, as well as finalize the value chain analysis and the value proposition of MAGNETO.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The ambition of the MAGNETO project is to empower LEA analysts and investigators with tools and infrastructures not commercially available today, to handle the petabytes of data that must be processed during an investigation. The solutions, tools and infrastructures offered by MAGNETO will provide important advances beyond the state-of-the art, which include:
-Multimedia analysis: MAGNETO aims to perform content and context metadata extraction, data indexing, data fusion and efficient search and retrieval.
-Semantic information fusion: MAGNETO aims to build a comprehensive Semantic Information Fusion system based on recent advances, to deploy it at the test sites, and to implement an ontology modelling the classes and their interrelations for the data streams arriving at the information fusion stages.
- Text mining, data mining, pattern recognition and machine learning: In MAGNETO, semantic notations and real-time machine learning algorithms will enable efficient processing and analysis in terms of speed, variety and different types of content.
- Heterogeneous data exchange and sharing: MAGNETO's ontology will allow knowledge exchange between different LEA systems.
- Augmented intelligence: MAGNETO investigates novel immersive Human-Machine Interfaces (HMIs), based on web and virtual reality technologies, creating a multidimensional environment that allows better situational awareness.

The MAGNETO project is expected to deliver the following results:
- Evidence collection platform employing heterogeneous data mining and multimedia content indexing making it available for further processing.
- Sophisticated representational model for representing knowledge in an open, standardized manner to unlock the discovery of relations and enable interoperability.
- Advanced correlation engine, which will facilitate automatic discovery of new, unsuspected relations within big datasets.
- Threat prediction engine by semantic reasoning, which will enable trends analysis and threat level assessment.
- Augmented intelligence tools that will deliver immersive and interactive HMIs to enhance situation awareness.
- Results transferability framework.
- Legal and ethical assessment framework for examining the impact of MAGNETO and ensuring its compliance to ethical standards and legislation.
- Training curricula for the use of MAGNETO tools.

MAGNETO brings a set of substantial impacts including:
- Improvement of investigation capabilities of the LEAs, by providing tools that reduce the time to solve crimes.
- Enhanced prediction and prevention of terrorist endeavors.
- Better identification and understanding of criminal activities.
- Better capabilities to prevent, investigate, and mitigate criminal acts.
- Compliance with the Privacy and Ethical directives of EU.

MAGNETO Concept

MAGNETO Exploitation

MAGNETO Ethics-Legal Overview