Skip to main content

InterCONnected NEXt-Generation Immersive IoT Platform of Crime and Terrorism DetectiON, PredictiON, InvestigatiON, and PreventiON Services

Periodic Reporting for period 1 - CONNEXIONs (InterCONnected NEXt-Generation Immersive IoT Platform of Crime and Terrorism DetectiON, PredictiON, InvestigatiON, and PreventiON Services)

Reporting period: 2018-09-01 to 2020-02-29

The continuous evolution of serious and organised crime and terrorism poses a challenge for Law Enforcement Agencies (LEAs) that need to provide effective countermeasure solutions for the detection, prediction, investigation, and ultimately prevention of such activities. In particular, the challenges for LEAs are to: (a) effectively correlate data from multiple heterogeneous sources during all stages (intelligence gathering, surveillance, operation, and investigation) of the process; (b) enable efficient access to the multimodal information pertinent to the different stakeholders involved; (c) efficiently simulate in 3D sites of interest in an accurate and visually convincing manner so as to facilitate operational and tactical planning and/or assist investigation; and (d) support the path-to-court of digital evidence.
CONNEXIONs aims to equip LEAs with an interconnected suite of advanced next-generation technologies for significantly improving their capabilities to gather intelligence, analyse evidence, and investigate crime and terrorism effectively and efficiently, with particular focus on improving the capabilities of officers across the board by encompassing the entire lifecycle of law enforcement operations, including: (a) pre-occurrence crime prediction and prevention; (b) during-occurrence LEA operations; and (c) post-occurrence investigation, and crime-scene simulation and 3D reconstruction.

To this end, CONNEXIONs builds upon two main concepts: (1) the multidimensional integration and correlation of heterogeneous multimodal data, including Web and social media multilingual and multimedia content, data acquired by Internet of Things (IoT) devices, police reports, and digital evidence, and (2) the delivery of pertinent information to various stakeholders in an interactive manner tailored to their needs, including through immersive (augmented reality - AR and virtual reality - VR) environments.

The main CONNEXIONs results will be: (1) Multimodal information extraction, understanding, integration, correlation, retrieval, and delivery services; (2) Immersive environments for improving situational awareness, investigation and training capabilities; (3) Operation Command Centre for real-time management of LEA operations; and (4) Integrated platform based on ethics and privacy by-design principles, implementing EU legal requirements, whilst being highly customisable to local legislation.

The developed CONNEXIONs tools and solutions will be validated by the end users in field demonstrations in three operational pilot use cases: (1) counter-terrorism security in large scale public events; (2) human trafficking investigations and mitigation, and (3) crime investigation and training through 3D scene reconstruction.
During the first half (M1-M18) of the project, CONNEXIONs achieved all the objectives and reached all milestones set for this reporting period. First, by following a user-centred approach, a study on data and resources for supporting the use cases was performed and the user requirements were defined collaboratively by the LEAs in the consortium, based on which the CONNEXIONs technical specifications were set and the platform architecture was designed. In addition, the initial plans for the implementation and evaluation of the field demonstrations of the three use cases were devised, along with a training plan.

The CONNEXIONs architecture was first realised by an operational prototype, which integrates the skeleton versions of the technical modules, and then by the first prototype which integrates the initial versions of the following components: (a) Discovery & acquisition of relevant Web and social media content; (b) IoT data normalisation & propagation; (c) Multilingual textual content transcription, translation, and analysis; (d) Visual understanding of multimedia data; (e) Multimodal analytics, textual generation, and threat assessment; (f) Μultimodal retrieval and semantic reasoning; (g) 3D reconstructions and immersive AR/VR applications; and (h) Operation Command Centre for real-time management of operations.

Dissemination and collaboration activities also progressed during the first reporting period, including the participation in (and organisation of) events, conferences and workshops, the publication of scientific papers, the generation of dissemination material, the setting up of a website and social media presence, the creation of synergies with related projects and initiatives, and the establishment of a stakeholder network. Moreover, an initial market analysis, exploitation plan, and business models were also delivered. Finally, the ethical, legal, and security guidelines and processes were set up, together with an overall framework for collaboration.
CONNEXIONs aims to equip LEAs with an interconnected suite of advanced next-generation technologies for significantly improving their capabilities to gather intelligence, analyse evidence, and investigate crime and terrorism effectively and efficiently.

CONNEXIONs will thus achieve the following impacts: (a) Provision of better tools tailored to the needs of LEA officers across the board to support them on their daily work by encompassing the entire lifecycle of law enforcement operations; (b) Prevention of more terrorist endeavours both at the investigation stage also and at the operation stage during specific missions; (c) Better identification and understanding of criminal and terrorist activities based on enhanced multimedia and multilingual content analysis for the automatic detection, classification, and tracking of objects and activities of interest; (d) Improved investigation and training capabilities through advanced next-generation technologies; and (e) Rapid crime solving that has the potential to reduce societal distress, investigative costs, and impact on victims / relatives.

The CONNEXIONs expected final result is an interconnected suite of advanced next-generation technologies that aim to improve the state-of-the-art in several scientific and technological fields, thus facilitating the implementation of the promised impact; these include:
(1) Discovery and acquisition of relevant multimedia and multilingual online content, and the subsequent multilingual automatic speech recognition, machine translation, and analysis of the acquired textual material in the languages of interest.
(2) Acquisition, extraction, fusion, analysis, and visualisation of multimodal data obtained from diverse IoT devices and sensors.
(3) Visual understanding and extraction of useful information from multimedia data.
(4) Semantic integration, enrichment, and reasoning on the basis of the CONNEXIONs ontology used for the semantic representation of available data and extracted information.
(5) Multimodal information correlation for extracting valuable knowledge pertinent to the LEAs’ information needs out of the high volume of available data while uncovering the structure underlying the data, and for assessing potential threats.
(6) 3D reconstruction and immersive (AR and VR) technologies for improving situational awareness and investigation capabilities
(7) Operation command centre for real-time operation management and command capabilities for LEAs during law enforcement and public safety operation.
CONNEXIONs logo