Periodic Reporting for period 2 - PREVISION (Prediction and Visual Intelligence for Security Information)
Periodo di rendicontazione: 2020-09-01 al 2021-12-31
Organised Crime Groups quickly adopt and integrate new technologies into their ‘modi operandi’ or build brand-new business models around them. More than 5,000 OCGs operating on an international level are currently under investigation in the EU, whereas document fraud, money laundering and the online trade in illicit goods and services are recognised as the engines of organised crime. Notably, goods and services offered on the Darknet are available to anyone, be it an individual user, an OCG or terrorist organisation. Almost all types of organised crime, criminals are deploying and adapting technology with ever greater skill and to ever greater effect, which represents the greatest challenge being posed to law enforcement authorities in the EU and globally at present time. Exponentially growing connectivity of all types of devices, including phones and appliances, is expected to amplify this concern, as criminals are already deploying techniques to exploit their vulnerabilities in order to gain access to personal and financial information or confidential business data. All of the above can have tremendous impact in the society. The effects on victims of terrorism and organised crime can be devastating and multiple and the consequences can be experienced at many interrelated levels - individually, collectively and societally. They often go far beyond the deaths, injuries, destruction and direct monetary losses and extend to the psychological effects on the population, the indirect financial costs, as well as the social and political impact. So, PREVISION’s objective is to provide LEAs with the capabilities of a) analysing and jointly exploiting multiple massive data streams, coming from online social networks, the open web, the Darknet, CCTV and video surveillance systems, traffic and financial data sources, and many more, b) semantically integrating them into dynamic knowledge graphs that capture the structure, interrelations and trends of terrorist groups and individuals, cybercriminal organisations and OCGs, c) predicting abnormal or deviant behaviour and radicalisation risks, based on sound predictive policing methods, underpinned by valid psychological, sociological and linguistic models, in conjunction with historical data patterns, d) performing dependable soft target risk assessment and cybercrime trend prediction at different timescales, e) becoming continuously more knowledgeable of the operations and activities of criminal organisations, by coupling the semantic technologies with deep and ensemble learning techniques, f) maintaining high situation awareness at all times by means of user-centred visual analytics and human-machine interaction techniques.
PREVISION has achieved a set of important objectives and tangible results:
a) A collection of over 20 data analytics tools, capable of processing different types of heterogeneous data sources and over 37 different types of data (including video, web, darkweb, telecom, network, financial, traffic, and others).
b) A powerful ontology comprising almost 1000 different classes for representing crime events and actors, combined with an extensible Semantic Reasoner capable of fusing and inferring semantic information.
c) An integrated and scalable platform that incorporates and links all developed tools together, allowing to compose workflows and service bundles
d) A common web-based Human-Machine Interaction environment, integrating and harmonizing the visual outputs of different tools.
e) A high-TRL tool for crawling, analysis and detection of stolen cultural objects, called ARTE-Fact.
f) Demonstration and evaluation of developed functionalities in 5 representative use cases (anti-radicalization, protection of public spaces, fighting of cyber-enabled crime, identification of fraudulent companies, mitigating illicit trafficking of cultural goods) in multiple iterations across different phases of the project.
g) Organization of a series of 17 online, physical or hybrid workshops for demonstration, feedback and evaluation.
h) Establishment of an online training portal.
i) Clustering and collaboration with 12 other security research projects, publication of 17 peer-reviewed papers, production of 1 white paper on knowledge engineering, and participation in over 25 dissemination events.
j) Development and application of comprehensive ethics guidelines, and publication of a multi-stage societal acceptance study as a policy paper.
a) A collection of over 20 data analytics tools, capable of processing different types of heterogeneous data sources and over 37 different types of data (including video, web, darkweb, telecom, network, financial, traffic, and others).
b) A powerful ontology comprising almost 1000 different classes for representing crime events and actors, combined with an extensible Semantic Reasoner capable of fusing and inferring semantic information.
c) An integrated and scalable platform that incorporates and links all developed tools together, allowing to compose workflows and service bundles
d) A common web-based Human-Machine Interaction environment, integrating and harmonizing the visual outputs of different tools.
e) A high-TRL tool for crawling, analysis and detection of stolen cultural objects, called ARTE-Fact.
f) Demonstration and evaluation of developed functionalities in 5 representative use cases (anti-radicalization, protection of public spaces, fighting of cyber-enabled crime, identification of fraudulent companies, mitigating illicit trafficking of cultural goods) in multiple iterations across different phases of the project.
g) Organization of a series of 17 online, physical or hybrid workshops for demonstration, feedback and evaluation.
h) Establishment of an online training portal.
i) Clustering and collaboration with 12 other security research projects, publication of 17 peer-reviewed papers, production of 1 white paper on knowledge engineering, and participation in over 25 dissemination events.
j) Development and application of comprehensive ethics guidelines, and publication of a multi-stage societal acceptance study as a policy paper.
PREVISION greatly advanced the capabilities of Law Enforcement Agencies (LEAs) in dealing with vast and heterogeneous amounts and streams of data, providing contributions into various stages of the intelligence cycle, from data crawling and collection up to visual analytics and decision support.
Starting from the data layer, PREVISION allowed the use of data crawling functionalities in a secure, controlled and supervised manner, enabling investigation of web and darkweb seeds. A flexible Extract-Transform-Load (ETL) component enabled the transformation of various data types into a common data representational model implemented in the form of an ontology. A series of individual analytics tools worked on various modalities to extract useful and meaningful information. For instance, in terms of video analytics, PREVISION advanced on object detection and tracking by investigating a novel hybrid representation of shallow and deep representation features. For action recognition, the goal-based descriptors were extended with spatiotemporal texture. For crisis event detection, PREVISION investigated the implementation of spatio-temporal techniques under a deep learning scheme for the detection of crisis events in near-real time. For face recognition, PREVISION leveraged facial points detection and a combination of shallow features with a deep convolutional framework. In terms of text analytics, PREVISION produced advances in the domains of text feature extraction, jargon detection, text mining and radicalization detection. In the domain of social media analytics, PREVISION developed novel community detection, key actor identification and actor identity resolution functionalities which were tested using publicly available datasets. Cyber analysis tools were also incorporated, making steps for the mitigation of hybrid security threats. PREVISION also developed a toolkit to identify radicalization tendencies in society at an early stage and, based on this, to develop risk forecasts that allow security authorities to take preventive action. At the semantic level, by developing a powerful Semantic Reasoner, it proved possible to implement complex queries, revealing hidden patterns, correlations and anomalies within vast amounts of data. Visual analytics were facilitated through the design of a modern web-based HMI, enabling different forms of visualization and user interaction.
PREVISION has established an open and future-proof platform for providing cutting-edge practical support to LEAs and practitioners in the fight against terrorism, organised crime and cybercrime, creating a number of impacts on societal and innovation aspects, such as:
1. Contribution to the Security Union.
2. Innovation capacity and integration of new knowledge.
3. Strengthening the competitiveness and growth of companies.
4. Societal impact(s), including alleviation of first/second/third order victimisation and reduction of financial costs.
Starting from the data layer, PREVISION allowed the use of data crawling functionalities in a secure, controlled and supervised manner, enabling investigation of web and darkweb seeds. A flexible Extract-Transform-Load (ETL) component enabled the transformation of various data types into a common data representational model implemented in the form of an ontology. A series of individual analytics tools worked on various modalities to extract useful and meaningful information. For instance, in terms of video analytics, PREVISION advanced on object detection and tracking by investigating a novel hybrid representation of shallow and deep representation features. For action recognition, the goal-based descriptors were extended with spatiotemporal texture. For crisis event detection, PREVISION investigated the implementation of spatio-temporal techniques under a deep learning scheme for the detection of crisis events in near-real time. For face recognition, PREVISION leveraged facial points detection and a combination of shallow features with a deep convolutional framework. In terms of text analytics, PREVISION produced advances in the domains of text feature extraction, jargon detection, text mining and radicalization detection. In the domain of social media analytics, PREVISION developed novel community detection, key actor identification and actor identity resolution functionalities which were tested using publicly available datasets. Cyber analysis tools were also incorporated, making steps for the mitigation of hybrid security threats. PREVISION also developed a toolkit to identify radicalization tendencies in society at an early stage and, based on this, to develop risk forecasts that allow security authorities to take preventive action. At the semantic level, by developing a powerful Semantic Reasoner, it proved possible to implement complex queries, revealing hidden patterns, correlations and anomalies within vast amounts of data. Visual analytics were facilitated through the design of a modern web-based HMI, enabling different forms of visualization and user interaction.
PREVISION has established an open and future-proof platform for providing cutting-edge practical support to LEAs and practitioners in the fight against terrorism, organised crime and cybercrime, creating a number of impacts on societal and innovation aspects, such as:
1. Contribution to the Security Union.
2. Innovation capacity and integration of new knowledge.
3. Strengthening the competitiveness and growth of companies.
4. Societal impact(s), including alleviation of first/second/third order victimisation and reduction of financial costs.