Skip to main content

Prediction and Visual Intelligence for Security Information

Periodic Reporting for period 1 - PREVISION (Prediction and Visual Intelligence for Security Information)

Reporting period: 2019-09-01 to 2020-08-31

Organised Crime Groups quickly adopt and integrate new technologies into their ‘modi operandi’ or build brand-new business models around them. More than 5,000 OCGs operating on an international level are currently under investigation in the EU, whereas document fraud, money laundering and the online trade in illicit goods and services are recognised as the engines of organised crime. Notably, goods and services offered on the Darknet are available to anyone, be it an individual user, an OCG or terrorist organisation. Almost all types of organised crime, criminals are deploying and adapting technology with ever greater skill and to ever greater effect, which represents the greatest challenge being posed to law enforcement authorities in the EU and globally at present time. Exponentially growing connectivity of all types of devices, including phones and appliances, is expected to amplify this concern, as criminals are already deploying techniques to exploit their vulnerabilities in order to gain access to personal and financial information or confidential business data. All of the above can have tremendous impact in the society. The effects on victims of terrorism and organised crime can be devastating and multiple and the consequences can be experienced at many interrelated levels - individually, collectively and societally. They often go far beyond the deaths, injuries, destruction and direct monetary losses and extend to the psychological effects on the population, the indirect financial costs, as well as the social and political impact. So, PREVISION’s objective is to provide LEAs with the capabilities of a) analysing and jointly exploiting multiple massive data streams, coming from online social networks, the open web, the Darknet, CCTV and video surveillance systems, traffic and financial data sources, and many more, b) semantically integrating them into dynamic knowledge graphs that capture the structure, interrelations and trends of terrorist groups and individuals, cybercriminal organisations and OCGs, c) predicting abnormal or deviant behaviour and radicalisation risks, based on sound predictive policing methods, underpinned by valid psychological, sociological and linguistic models, in conjunction with historical data patterns, d) performing dependable soft target risk assessment and cybercrime trend prediction at different timescales, e) becoming continuously more knowledgeable of the operations and activities of criminal organisations, by coupling the semantic technologies with deep and ensemble learning techniques, f) maintaining high situation awareness at all times by means of user-centred visual analytics and human-machine interaction techniques.
During the first reporting period, PREVISION fully covered its objectives and completed its milestones and deliverables. Not only did it accomplish a set of significant technical outcomes, but it also strengthened its legal and ethics aspects, as well as conducted a comprehensive series of communication and exploitation activities, reaching a wide audience and establishing a concrete methodology of developing its business plan. The efforts of PREVISION consortium in this first period focused on the design of a reliable modular and fully interoperable platform that is at the same time innovative and affordable, so to allow LEAs to solve complex cases faster and more efficiently. To this aim, the partners of PREVISION have carried out different actions to
a) worked towards delivering a set of the first versions of all data streams analytics tools. In particular, crawling tools have been developed for extreme-scale data collection, while for the processing of the heterogeneous data sources the developed tools include tools for performing visual and textual analysis, as well as tools for analysing social networks.
b) more than 50 biographies of known terrorists and extremists were collected in order to gain basic knowledge about their behavior and develop the corresponding analysis models for the detection and modification of radicalization processes.
c) own taxonomies and keywords had to be developed and created that are necessary to analyze corresponding findings from texts, voice and video files and to be able to feed them into a scoring system.
d) trend and network analyses will be applied in the areas of cybercrime and organized crime. This will make it possible to identify and analyze networks of relationships between natural persons, legal entities and companies/organizations on the basis of personal or public data.
e) a list of tools was identified, that potentially constitute competition for PREVISION situational awareness applications.
f) the infrastructure for installing the PREVISION platform has been configured and the first architectural version of the platform was defined.
g) simulated/synthetic data for testing has been prepared and testing was performed for routing and processing of simulated data.
h) ensuring compliance with the relevant ethical and legal standards. Ethical and legal guidelines for the development and use of tools. An Ethics Review Panel has been formed that consists of six experts (three external and three internal to the consortium) and provides advice on ethical and fundamental rights issues that occur in the project.
PREVISION advances on object detection and tracking by investigating a novel hybrid representation of shallow and deep representation features. For action recognition, the goal-based descriptors will be extended with spatiotemporal texture. For crowd analysis, PREVISION combines swarm-based descriptors with a deep representation. For crisis event detection, PREVISION investigates the implementation of spatio-temporal techniques under a DeepCNN scheme for the detection of crisis events in a near-real time manner. For face recognition, PREVISION leverages facial points detection and a combination of shallow features with a deep convolutional framework.
Regarding steganographic traffic detection, PREVISION will process network traffic traces with and without covert traffic in conjunction with information that such traces contain (or not) secret data.
With respect to information fusion, PREVISION addressed three main challenges using Markov logic networks, the data association problem, the need for a high quality statistical model, which must be trained using sufficient labelled training data, and the adaptation of the logical (object oriented) model during its usage.
PREVISION also develops a toolkit to identify radicalization tendencies in society at an early stage and, based on this, to develop risk forecasts that will allow security authorities in advance to take preventive action. This requires a continuous observation of social processes and especially of violent political milieus.
PREVISION establishes an open and future-proof platform for providing cutting-edge practical support to LEAs and practitioners in the fight against terrorism, organised crime and cybercrime, creating a number of impacts on societal and innovation aspects, such as:
1. Contribution to the Security Union.
2. Innovation capacity and integration of new knowledge.
3. Strengthening the competitiveness and growth of companies.
4. Societal impact(s), including alleviation of first/second/third order victimisation and reduction of financial costs (estimated at an astonishing $600 billion in 2017).
architecture.png
methodology.png