Final Report Summary - LASIE (LArge Scale Information Exploitation of Forensic Data)
Executive Summary:
LASIE will offer a novel framework to assist law enforcement agencies and investigators in their everyday work. The proposed system consists of tools for automatic analysis of vast amounts of heterogeneous forensic data (text, images, video, audio and biometric information) acquired from different sources including CCTV surveillance content, confiscated desktops and hard disks, mobile devices, Internet and social networks. The framework guides the investigation procedure through the incorporation of recommender services and interacts with users though efficient and user-friendly graphical user interfaces. The envisaged framework is expected to significantly reduce the required investigation time as well as to have a positive impact not only on the actual level of security but also on how security is socially perceived resulting in an improved daily quality of life especially in the social environment.
The main expected outcomes from LASIE project will be:
• A set of tools for automated processing of forensic data including text, images, video, audio and biometric information, able to analyse and fuse vast amounts of heterogeneous data acquired from different sources.
• Novel inference and recommendation mechanisms that will guide the analysts through their investigation. These mechanisms will exploit appropriate knowledge modelling procedures, taking also into account explicit analyst knowledge and information from previous cases, which will help analysts to manage the results of the automated processing.
• Tools able to link and merge information from heterogeneous sources. This will be achieved by exploiting an event-oriented approach that will offer a uniform representation of information irrespective of the underlying data types.
• An appropriately designed user interface able to automatically highlight relevant events. The user interface will allow the analyst to create his/her own cases in a user-friendly graphical way. Additionally, the analyst will be able to define his/her entities and events, visualize causal, temporal and spatial sequences and test or refine different hypotheses.
• A framework for evaluation through simulated procedures in order to have quantitative measures of the system results.
• A series of training sessions to analysts and operators to make them capable of using it efficiently and with ease.
All the above will be realised following a privacy-by-design approach to ensure privacy protection through built-in, rather than added-up, legal and ethical compliance mechanisms. It is worth mentioning that Legal, Ethical and Privacy (LEP) negotiation will be provided both per module and in a system level.
Project Context and Objectives:
The main goal of LASIE is to design and implement an open and extendable framework that will significantly facilitate the investigation process of the analysts, providing them automatic analysis services of vast amounts of heterogeneous forensic data. This framework guides the investigation procedure through the incorporation of recommender services and interacts with users though efficient and user-friendly graphical user interfaces. LASIE will be performed under the condition that all legal and ethical restrictions are satisfied and the computed evidence data can be presented in European courts of law.
The main LASIE objectives can be broken down into the following 11 objectives:
1) Tools for supporting automated processing of very large amounts of data (WP4): An efficient way to address the processing of large datasets is to reduce the search space by applying advanced media analysis techniques, fusion of information and filtering based on time/space/event/pattern dimensions of the information. LASIE will develop and test advanced media analysis tools for the different types of content (text, audio, images, video, social and biometric data, etc.). These tools will extract relevant low-level and high-level features to be used in the next steps of the investigation. The objective of the 3rd reporting period is the finalisation of the forensic data processing modules that were developed during the 2nd reporting period (further improvements based on the comments of the 2nd review). The goal is to prepare the final versions for integration in LASIE platform.
2) Domain knowledge acquisition and modelling (WP5): In order to enable an efficient investigation, information gathered from multiple and heterogeneous sources has to be transformed into usable knowledge. Such a transformation will involve the use of analysis tools and modules to extract mid-level semantics from the forensic data. Upon the extraction, with the help of generic intelligence knowledge representation schemas, appropriate knowledge structures will be developed. In LASIE a comprehensive knowledge representation framework for modelling, indexing, classification and retrieval of data for forensic applications will be developed. The knowledge representation framework will integrate diverse sources of information including geospatial grid representation, temporal semantics, event representation and object annotation through a multimedia ontology. During the 3rd repotting period, the objective is the finalisation of the LASIE knowledge representation framework and tools for event-based modelling and extracting data patterns and anomalies.
3) Development of an Intelligent Evidence-based Search Engine (WP6): The search engine will enable search and retrieval in both the existing forensic databases and the various captured forensic data. The searchable data types comprise text (in the form of documents, email, internet history, calligraphic, etc.), images (faces, tattoo, plates, etc.), video (stored in local disks, surveillance, etc.), audio (speech, emotions, etc.), social and biometric data, i.e. all types of forensic data. Due to the high heterogeneity of these data types, several search modules will be implemented, each one adapted to the specific technical constraints of each type. The individual search modules will exploit technologies for distributed processing and large-scale indexing to enable fast and efficient search even to vast repositories of forensic content. Indexing will be performed on the metadata extracted with the processing tools. The objective of the 3rd reporting period is to implement the LASIE multimodal search engine that will integrate individual search modules, the final version of the complex query formulation and relevance feedback. Moreover, the finalization of the crowdsourcing tool for super recognisers is one of the main objectives within the WP6.
4) Provision of guidance to analysts through inference and recommendation functionalities (WP7): The Recommendation and Inference modules will be designed and developed within LASIE to achieve the following objectives:
• To continuously provide possible investigation directions (e.g. through the presentation of information relevant to the already provided evidence data) to the analyst, who will eventually decide whether to pursue them or not.
• To perform automated inference based on the provided evidence, in order to detect and resolve possible inconsistencies or to reduce uncertainty associated with the estimated analysis results.
• To highlight important information, suggest relevant information sources and supply additional information regarding a particular entity, based on its relations to other entities.
The objectives of the 3rd reporting period are:
• To finalise the development of the recommendation module and integrate it in LASIE framework.
• To finalise the development of the inference module and integrate it in LASIE framework.
• To update the Evidence graph (EG), integrating new developments of WP4 and WP6 modules.
5) User Interface that enables Presentation of Highlights and Refinement of Investigation Process (WP8): LASIE will provide visualization of the results in order to present relevant highlights addressing the who, where, when, what and why, and give the analyst the capability to refine his/her search queries and hypotheses. LASIE will develop a user interface specifically designed for forensic purposes. The user interface will provide modelling and simulation in order to test operational procedures and techniques through visual analytics and crime scene reconstruction modules. Special focus will be given on its design in order to produce a user-friendly GUI that can be easily manipulated even by non-IT expert police officers. The objective of this period is the development of updated LASIE User Interface, by integrating query formulation, visual analytics and crime scene reconstruction, taking also into account feedback/recommendations from the previous version.
6) Legal and Ethical Compliance through Privacy by Design (WP2): The LASIE framework will apply a Privacy-by-Design methodology, providing built-in privacy protection mechanisms rather than added-up security. The objective of the 3rd reporting period is to follow the methodology to encure legal and ethical compliance during the design and development of all LASIE modules, as well as development of the ethics and digital evidence (EDE) module.
7) Continuous integration and testing (WP9): Following the system requirements and architecture (WP3), the objective of the 3rd reporting period is to implement the final version of the LASIE platform by integrating the final versions of all LASIE modules. Moreover, the system testing and validation is planned for this period.
8) Pilots and user evaluation (WP10): An objective of the 3rd reporting period is also to prepare the final arrangements of the LASIE pilots at MET and ADM premises, to run the pilots using the final integrated version of the LASIE platform and prepare a framework for user evaluation thourgh questionnaires appropriately designed to evaluate the user requirements.
9) Project dissemination and training (WP11, WP12): LASIE dissemination aims to maximise social, scientific and industrial visibility of the developments, findings and outcomes of the project. Apart from disseminating LASIE through industrial fairs, workshops/conferences, paper and internet media, publications in international journals and conferences, it is important from the very beginning of the project to:
• setup and maintain the project website
• prepare LASIE Newsletter (updated every six months)
• setup a LASIE Advisory Board, composed of end-users and security authorities with the aim to support the Consortium in the definition of operational requirement, scenario prioritization, testing, validation and assessing of the crowd management solutions and methods adopted and developed in the project.
Regarding training activities, an objective of this reporting period is to organize the third LASIE training workshop and prepare the supporting training material.
10) Exploitation of project results (WP11): The objective of the 3rd reporting period is to finalise the market analysis and exploitation plan that were initialised during the previous reporting period. The exploitation plan shall include the tools that will be available by the end of the project.
11) Contribution to Standards (WP11): The objective of this period is to proceed to the schedule structured within the previous reporting period and finalise the standardization activities based on this schedule. The activities should be aligned to the plan of the research conducted in WP4-8.
Project Results:
A summary of the work and main results achieved during the third reporting period are summarized below:
1) Tools for supporting automated processing of very large amounts of data (WP4): All planned modules for processing of forensic data were finalized and integrated during the third reporting period. More specifically, in text processing, new services were implemented for dealing with Jargon and specific terms used mainly in the Social Media contents; in image processing, new modules were implemented for logo detection, face detection as well as the EXIF module, following the recommendations of the second review; regarding video processing, the modules for crowd violence, group detection and people tracking were finalized and demonstrated in MET data. Morever, all social media crawlers have been finalized and provided for integration as services outputting a valid JSON object containing the retrieved content. All related updates have been reported in reported in Deliverable D4.7 Final version of LASIE forensic data processing tools (M24). D4.7 includes further details on minimum resolution for face and plate detection.
2) Domain knowledge acquisition and modelling (WP5): The most important result in this area has been to move the knowledge acquisition from surface text to deep semantics, which in many cases is very different from the surface appearance. This novel step is crucial in order to be able to reason correctly about the data. The LASIE Knowledge Representation Repository was improved, in order to achieve an ontological model to integrate all heterogeneous data. Moreover, a refinement of existing supportive ontologies (Context, Category, Entity, etc) based on UC2 proposed scenario took place. The visualization toolkit was developed by SEN in order to visualise immediately the most important aspects of the story, namely its causal and temporal sequence, and in its ability to restrict or expand the views of events and entities. Finally, the knowledge structure for LASIE event-based knowledge representation was improved, achieving a modular ontological model for LASIE forensic domain. An Ontology Manager was defined and developed, which provides the main functionalities to manage ontologies through API. The definition of recurrent knowledge graphs represents common complex information to store into knowledge base, to retrieve information in a more easy and complete way. The work in all the above activities is reported in Deliverables D5.3 Knowledge Representation Repository and supporting tools (Update of D5.2) and D5.4 Report on Event-based Knowledge Modelling (M34).
3) Development of an Intelligent Evidence-based Search Engine (WP6): One of the most significant results, during the 3rd reporting period, is the finalization of the LASIE multimodal search engine, which integrates the latest updates in individual search (text, image, audio, video) modules, the Complex Query Formulation (CQF) and the Relevance Feedback (RF). The achievements are reported in Deliverable D6.3 LASIE multimodal search engine (2nd iteration) (M34). Moreover, the TooManyEyes Crowdsourcing tool for Super-recognizers was finalized after collaboration between CERTH and UoG. The outcome was a tool of high quality, thus, an initial plan for commercial exploitation of the tool has been prepared. The achievements are reported in Deliverable D6.4 Mechanisms for relevance feedback and human-driven search refinement (M34).
4) Provision of guidance to analysts through inference and recommendation functionalities (WP7): During the 3rd reporting period, the achieved results in objective 6 are listed below:
• The Evidence Graph (EG) has been updated based on the feedback of the previous reporting period, in order to integrate new features.
• The LASIE recommendation module has been finalised and integrated in LASIE framework, supporting all three Use Cases. This work has been reported in Deliverable D7.4 Evidence Recommendation module (2nd iteration) (MS38).
• The LASIE inference module has been finalised and integrated in LASIE framework. A mechanism to retrieve relevant knowledge sub-graph. Moreover, a semantic approach, based on a reasoner, ontologies and rules, to create hypothesis for the proposed scenarios was defined and developed. This work has been reported in Deliverable D7.3 LASIE Inference Framework (2nd iteration) (MS33).
5) User Interface that enables Presentation of Highlights and Refinement of Investigation Process (WP8): An updated version of the LASIE User Interface was delivered within the 3rd reporting period, to support all three Use Cases in the pilot activities and the third LASIE Workshop. All recommendations from the 2nd review and user feedback were taken into account to improve the UI in several aspects. All modules, i.e. the Query Formulator (QF), the Visual Analytics and the Crime Scene Reconstrution (CSR) were integrated. The approach for assessment of the User Experience has been followed to ensure that the UI has been improved according to the recommendations. All the above work has been reported in Deliverable D8.3 The LASIE User Interface (2nd iteration). The work in Visual Analytics module and Crime Scene Reconstruction module has been included in D8.1 Video and Visual Analytics and D8.4 Interface for crime scene reconstruction, respectively.
6) Legal and Ethical Compliance through Privacy by Design (WP2): During the third reporting period, the information sharing agreements were finalised and DPAs were notified, where applicable. Moreover, the implementation of the ethics and digital evidence (EDE) module was finalised. The module ensures that both the requirements concerning privacy, personal data protection, ethics of surveillance and laws on digital evidence, as well as the principles of Responsible Research and Innovation (RRI), are adequately addressed in the LASIE prototype. Deliverables D2.3 Monitoring reports on emerging ethical challenges in the developing and implementing the system (Update of D2.2) (M42) and D2.5 Data control report (update) (M42) have been finalised and submitted by M42. Deliverables are updating the recommendations on how to take into consideration the societal issues raised by LASIE (data protection, ethics, privacy and so on), by analysing the technological landscape and change, legislative and societal change.
7) Continuous integration and testing (WP9): Most important results achieved in last months that impact on several aspects of the platform, are:
• Quality of data was improved through data filters that works transparently for the user, especially for crawlers and NLP modules;
• Modularization of the interpreter that processes the scripts;
• Implementation of new data fusion services that apply domain rules (based on ontology and data fusion strategies) that improves the quality of the knowledge base and allow to merge data that refers to the same entities but arrives fragmented and from several sources (e.g. data that arrives from two modules but refers to same event);
• Implementation of consistency rules and services that manage better relations between resources and entities (e.g. event creation starting from a video resource);
• Implementation of high level integration services (described in the D9.3) that simplify many operations performed by the GUI and that are based on the low level integration services (described in the D9.2);
• Simplification of the script language used to define the logic that describes how orchestration is done;
• Fully integration of knowledge patterns in order to ensure an high grade of consistency (an important issue in domain driven approach, see D5.4);
• Integration of ontological validation rule in all integration services and in general all new features provided by the knowledge management framework;
• Refinement of workflow with changes to existing integration services in order to meet some reviewers and users suggestions and to simplify the orchestration pipelines;
• Re-writing of some low level services in order to improve performances;
• Development of services for system monitoring and user action logs in order to improve the governance layer and meet some reviewers suggestions;
• The chosen methodology of integration of information extraction modules has demonstrated to respond well to the needs of the project;
• A new management of Hypothesis (WP7) at business logic was done.
Integration Framework also support GUI and integrate all module expected (as described by the WP3) are working. Prototype integration is actually completed. This work has been reported in Deliverable D9.3 Final Integrated LASIE framework (M42). According to the objectives of T9.4 NP worked together with ENG, CERTH, INNEN, UPM in order to elaborate a consistent testing and validation plan adapted to the identified needs of the overall LASIE framework. The validation plan was divided into two distinct processes: one focusing on validating each individual component against its requirements and one that was dedicated to the validation of the integrated LASIE platform. The validation and testing plan was guided by ISO 9126’s main principles for software quality assessment and its dimensions, subcategories and quality metrics were taken into account throughout the task activities. The first step for the individual validation of LASIE’s components was to determine the method by which each requirement would be validated. All consortium partners were involved in this process and a well-balanced mix of different validation methods has been chosen, based on requirements’ type and fit criterion. Finally, the technical partners of the consortium have led the technical validation activities and provided their findings for all the components that are involved in the LASIE’s Use Cases. For the validation of the integrated LASIE platform a set of quality metrics has been defined to monitor and assess the quality of the developed software based on ISO 9126 dimensions. The actual testing phase was carried out at the integrated LASIE prototype and targeted to confirm the quality of the software. LASIE platform was tested in an operational environment, and in selected test cases, thus, ensuring that the working blocks of the systems have been integrated successfully and work in conjunction. This work has been reported in Deliverable D9.4 LASIE technical validation (M42).
8) Pilots and user evaluation (WP10): The work in this WP for the 3rd reporting period includes the following:
• Successful realization of pilots in June 2017 in London (MET premises) and Madrid (ADM premises).
• Preparation of dedicated questionnaires and training materials for assessment and evaluations;
• Execution of evaluation session with internal and external end-users;
• Analysis of gathered feedback and production of D10.3.
The work has been reported in D9.2 Demonstration portal (M36) and D9.3 LASIE framework evaluation (M42).
9) Project dissemination and training (WP11, WP12): During the 3rd reporting period, the number of Advisory Board members has grown with a new expert (Mr Gianfranco Todesco, chief Commissioner of the Digital Investigations team at the Turin Local Police):
• Ten members from seven different countries (ES, GR, LT, PT, UK, FR, IT)
• Three members from international bodies
The consortium disseminated the project through discussions and presentations with technological partners and end users in three industrial events of the security market. The project was also disseminated within scientific conferences through high ranking at the DCASE conference challenge, a booth but also with the implementation of special sessions in ICIP and ICDP 2016 international events. Regarding scientific communications, two new journal papers and four new conference papers have been accepted for publication. Also, 7 talks/keynotes in universities and international conferences can be added to the dissemination listing. The project also organized two lunch seminars at PRIO premises and co-organized two workshops in London in the past months.
The work has been included in D11.3 – Report on Dissemination Activities (Update) (M42).
Two training sessions for end-users were organised in Madrid (21st June 2017) and London (30th June 2017) as well as one final training workshop in Turin (28th September 2017). In particular, the final workshop was held in Turin at the Turin University with the intermediation of the Advisory Board (in particular the Turin Local Police). The training was focused on practical individual sessions, where end users learned how to interact with the system. In Turin, the practical session was preceded by the explanation of the main features and on the usage of the system by end users. A roundtable of end users, researchers, Advisory Board members and representatives from sister projects concluded the workshop. The work has been included in D12.5 3rd Workshop Report (M42).
10) Exploitation of project results (WP11): With respect to exploitation, LASIE applied and approved to benefit from the Common Exploitation Booster support services offered by the European Commission H2020 Common Support Centre. EC has assigned to LASIE a coaching expert: Mr. CHICHOCKI Tomasz from META Group. Then, an Exploitation Strategy Seminar was organised in Oslo on June 13th, where all LASIE partners participated. Several results of the LASIE project were assessed and initial exploitation plans have been drafted by the involved partners. The work has been included in Deliverable D11.6 - Market analysis & Final Exploitation Plans (M42).
11) Contribution to Standards (WP11): In the current reporting period, VML participated in two MPEG meetings held at Chengdu, October 2016 and Geneva, January 2017 and contributed towards the development of Media Orchestration standard. The application of Audio-Visual patterns through the extraction of DROP features was included in the standards specification as normative reference. The bit stream representation for DROP features was standardised using the MPEG-B specification and in the standard, has been referenced as Region-of-Interest (ROI). The representation of the ROI uses the MPEG-B part 10 Amendment 1 (via ISO/IEC 23001-10) uses ISOBMFF file format for pointers to 2DCartesianCoordinateSamples, the 2DCartesianCoordinateSampleEntry, their timing and their relationship to video samples in the ISOBMFF file. Clause 6.2.2.1 specifies the syntax of a 2DCartesianCoordinateSample, which are 9 bytes in byte (8-bit) alignment. Zero or more of these 9-byte 2DCartesianCoordinateSample are contiguously contained inside an 'mdat' box. Clause 6.2.1.1 specifies the syntax of a 2DCartesianCoordinateSampleEntry, which is a MetadataSampleEntry of the type '2dcc'. Zero or one 2DCartesianCoordinateSampleEntry is contained inside a 'stsd' box.
The following input contributions were made on behalf of LASIE project to the MPEG MORE ad-hoc group. VML also developed a visualisation of the DROP based orchestration which was also presented in the LASIE review meeting. In the forthcoming period, VML will continue to contribute towards the reference software implementation.
Input contributions:
• Krishna Chandramouli, Panos Kudumakis, Ebroul Izquierdo, "LASIE Dataset for Media Orchestration”, ISO/IEC JTC1/SC29/WG11 (M39214), Oct 2016 (15/10/2016)
• Krishna Chandramouli, Panos Kudumakis, Ebroul Izquierdo, "Representation of Orchestration Data based on visual patterns”, ISO/IEC JTC1/SC29/WG11 (M40118-v4), Jan 2017 (14/01/2017)
The work has been reported in D11.7 - Report on standardisation efforts (M42).
Potential Impact:
LASIE impact on the “cost of the crime” and resolution of crimes:
The economic and social “cost of the crime” is an essential measure of the impact of the crime on the society. The cost of the crime gives us a way of measuring the impact of policies aimed at reducing crime and its consequences. Thus, the cost of crime has become an increasingly important tool for decision-makers concerned with crime’s impact on society. It helps make explicit judgments about the relative merits of alternative policies and programs, which are already implicit in decision-making about how to allocate resources to tackle crime – both overall and between different types of crime. According to the Home Office research report of October 2013, the social and economic costs of organised crime amount to many billions of pounds. Drugs supply (£10.7 billion), organised fraud (£8.9 billion) and organised immigration crime types (£1.0 billion) have major impacts on the UK, and other less visible crimes also cause substantial harm. This report outlines evidence on organised acquisitive crime types; organised child sexual exploitation; counterfeit currency; drugs supply; organised environmental crime; firearms; organised fraud; organised immigration crime; organised intellectual property crime; and organised wildlife crime, which all cause damage to the UK.
Moreover, crime volume in LASIE’s partners’ countries of origin (Germany, England, Wales, France, Italy, Belgium, Spain, Greece, Portugal, Norway, and Ireland) has reached 11.66 million police registered cases.
As far as the distribution of crime types in the LASIE’s partners’ countries of origin, it is evident that theft and burglary direct and indirect crimes occupy 80% of crime volumes, then followed by 10% assault crimes.
Moreover, if we take into account the total cost of the crime to the society, this can be much higher than the cost of the damages or the lost/damaged property. The property loss is only 10%, however the emotional and physical impact on victims is calculated to 50%, while response to crime by the criminal justice system constitutes 20% of the total cost of crime.
What is even more important is that only 1/4 of the crime is known to the police, less than 1/10 is cleared and only 1% is finally sanctioned (jail, fine, alternative punishment). Only in 2011, a third of crimes went unsolved, as police admitted there is little chance of a positive outcome.
Crime reduction and criminal policy is making progress but still, in the case of civil service tribunal, the number of pending case remains quite high, while in case of General Courts Tribunal the number is even increasing.
LASIE aims to significantly facilitate the investigation process of the analysts, by providing an automated initial analysis of the vast amounts of heterogeneous forensic data, guiding the investigation procedure through the incorporation of a recommendation functionality. This approach will not only increase the number of resolved cases and therefore reduce indirect cost of the crime, but will also lower the cost of investigation.
LASIE impact on legal and ethical issues:
Legal and ethical issues concern any investigation system since they are dealing with personal data. The principle is that "a person is innocent until proven otherwise". Yet, in many investigations, involving persons that are finally proven to have nothing to do with the case, may significantly harm them, their personality, their career, even their relatives.
LASIE carefully considers the legal, ethical and privacy issues. Experts within the project have studied these issues in D2.1 “Legal and ethical guidelines on data collection”. Moreover, the LASIE framework respects privacy and ethical issues by design. Such a result will increase public acceptance of next generation security system.
LASIE impact on increased (feeling of) security:
Fear of crime has become an important issue of public concern: a problem which detracts from the quality of life, and which adversely affects social and economic well-being. While the fear of crime expressed by some citizens is well-founded, other individuals are at less personal risk than they might believe. Their fear, however, is no less real. The levels of fear of crime in European neighbourhoods grow to reach 40% of the population in certain countries.
We expect that the increased effectiveness of LASIE as an investigation system will have a positive impact not only on the actual level of security but it will also positively affect the psychological feeling of security, resulting in a positive effect in every day quality of life especially in the social environment.
LASIE impact on increased trust in Police:
The trust in Police forces is also an important issue. A high level of trust is necessary in any democracy to ensure Police credibility and authority. It appears in different surveys that the citizens’ satisfaction rate of Police performance remains rather low. A recent survey in England and Wales reports that only two thirds of people think the Police is doing a good job. The satisfaction rate falls to only 39% in rural areas. Some other consequences may be a certain feeling of impunity to the criminals. By increasing the number of solved crimes and the delay in which they are solved, LASIE will indirectly participate in the increase of trust in Police forces.
LASIE impact on economic growth and job creation:
LASIE develops original state of the art know-how (and later on skills) in a specific area of investigation devoted to security systems applications. This area has significant economic potential itself. Yet, the LASIE results are achieved in the broader area of security and data analysis processing, which is a powerful driver of economic growth (and employment) in today knowledge-based economy. In short, LASIE will help to create opportunities in the specific area of investigation systems but a larger fall-out of some of its results can be envisaged in all areas where data processing can be harnessed to support security and innovation.
The market sector related to LASIE is the one related to forensic sector, in particular law enforcement investigators/analysts dealing with the analysis of different sources of forensic data coming from surveillance systems, voice recording devices, computer/mobile-phone network monitoring platforms, the networked social services. Such market is growing at a very high pace in recent years.
Modern societies are confronted with an increasing number of illegal events and crimes. Such extreme events are marked by threats to societies’ values and/or life-sustaining functions and create an urgent need to respond to them as quick as possible. In case of crimes, it is the responsibility of forensic sector to manage the investigation process in order to save lives and restore a sense of order. One of the constant challenges encountered by forensic data is the search in extremely large collections of heterogeneous forensic data, i.e. the information that is required for making high-quality decisions under pressure. Without accurate analysis, analysts/investigators find it hard to make fast and correct decisions. In fact, the absence of reliable evidences (combined with the flood of unsubstantiated reports and rumours) could jeopardize a fast resolution of the crimes. To solve such problems, LASIE develops appropriate (semi-) automatic tools for facilitating the analysts in their investigation activities. Such added value of LASIE will represent its unique selling proposition in the forensic sectors.
Due to a unique confluence of technological change and social demands, the forensics business is one of the most dynamic sectors of the modern economy. The capability of forensic techniques has grown, costs have dropped and hence decision makers have promoted forensics to a generally receptive public. The demand for products and services is therefore generally on an upswing. Based on research and survey, it is projected that total combined spending on forensic investigation by public and private sectors will increase year after year. Technology advancements in forensics have been of considerable import in recent years. BCC estimates that total sales of forensic products and services were nearly $10.1 billion in 2010 and that sales will grow at a 9.4% compound annual growth rate (CAGR) through 2016 to more than $17.2 billion. Other studies project that the US market for Forensic Technologies & Services will even register more than $20 billion in annual revenue by 2020.
LASIE, therefore, will provide its unique platform towards a market that is expected to increase its spending constantly, in an overall trend that is making security one of the fastest growing and challenging emerging business environment.
Key points for the commercial exploitation strategy has been determined making the basis for the route-to-market.
LASIE partners estimate that most of the expected exploitable results can be ready for commercial exploitation between 9 to 18 months after the project end.
At this stage of project development, the exploitation strategy planning foresees the following principles and route-to-market:
• The business models explored are licensing and SaaS, but other models will be taken into consideration according to the needs of different potential users and based on innovative adoption models. The starting assumptions are that the outcome of LASIE should be provided as a Software as a Service (SaaS), allowing end-users to sell the service to potential clients, rather than as a stand-alone software or perhaps as Cloud-solution for ultimate scalability and agility. We believe the real value of the test lies in the service provided by professionals who can use potential innovation on day-to-day activities which will spread information about LASIE in the forensics and security sectors.
• The Consortium is evaluating two main joint exploitation routes: (1) a joint-company is established amongst the LASIE partners, targeting the European market in collaboration, or (2) provide partners/end-users access to use the service, with centralized hosting, in effort to aggregate results from across the European market, making it possible to conduct in-depth research of human behaviour in advanced forensic and investigation activities, which may prove more valuable than the platform itself.
• LASIE platform can be exploited under three main packaging combinations: (1) light, (2) intermediate, (3) complete package. different packages can include different modules according to the real needs of the final users. The cost for each package will vary based on available modules.
• Importance to provide a customizable commercial solution able to guarantee different levels of needs for every customer, no matter which sector the target user is operating in and its company size;
• Starting from markets in Belgium, Germany, Ireland, Greece, France, Portugal, UK, Spain, Ireland and Italy, then all EU markets will be targeted in order to fulfil the business scenarios and expand the market presence of partners across Europe;
• A deep and detailed refining of the marketing and commercialisation plan need to be delivered later by specialized commercial Business Units containing all the activities to be carried out, containing strategies for acquiring customers and detailed services to be provided;
• Project partners aim to conclude some preliminary contracts with large and/or intermediate organisations, in the months right after the end of the project, in order to start the commercialisation phase as soon as possible or to find new funding opportunities for further product development. To this end, partners will leverage on their existing networks, counting mainly on public bodies and big organizations;
• The overall marketing strategies will be defined internally by skilled marketing and sales units and implemented by all companies by their business units. In particular, industrial companies through LASIE will empower their current product portfolio and will be able to provide it to their existing customers. Moreover, they are confident to boost their business opportunities also by selling to new customers also products and technologies already present in their current portfolio.
• A specialized long-term learning programme for the employees will be developed, on a periodical basis, addressed to the vulnerable groups. Furthermore, upon necessities, this programme will allow instant actions.
For all Key Exploitable Results (KERs) some common exploitation and commercialization issues arise:
The primary market of the consortium is a challenging one. Apart from the in-depth market and competition analysis the interested partners should better understand the decision-making process specific to this market and prepare the message for the key stakeholders. Alternatively, some of the results could be successfully used outside forensic and security field, such as broadcasting and entertainment industry which is less regulated and more market oriented. The consortium should take the advantage of Advisory Board members in finding the potential new customers.
The successful market introduction of project’s KERs will require significant financial investment. For relatively low TRL results the new Horizon2020 project could be a proper path for the development. The industrial partners may benefit form Horizon2020 SME Instrument and support of Enterprise Europe Network. Partners looking for private funding may use the European Investment Project Portal (EIPP) to look for the potential investors.
List of Websites:
http://www.lasie-project.eu/
LASIE will offer a novel framework to assist law enforcement agencies and investigators in their everyday work. The proposed system consists of tools for automatic analysis of vast amounts of heterogeneous forensic data (text, images, video, audio and biometric information) acquired from different sources including CCTV surveillance content, confiscated desktops and hard disks, mobile devices, Internet and social networks. The framework guides the investigation procedure through the incorporation of recommender services and interacts with users though efficient and user-friendly graphical user interfaces. The envisaged framework is expected to significantly reduce the required investigation time as well as to have a positive impact not only on the actual level of security but also on how security is socially perceived resulting in an improved daily quality of life especially in the social environment.
The main expected outcomes from LASIE project will be:
• A set of tools for automated processing of forensic data including text, images, video, audio and biometric information, able to analyse and fuse vast amounts of heterogeneous data acquired from different sources.
• Novel inference and recommendation mechanisms that will guide the analysts through their investigation. These mechanisms will exploit appropriate knowledge modelling procedures, taking also into account explicit analyst knowledge and information from previous cases, which will help analysts to manage the results of the automated processing.
• Tools able to link and merge information from heterogeneous sources. This will be achieved by exploiting an event-oriented approach that will offer a uniform representation of information irrespective of the underlying data types.
• An appropriately designed user interface able to automatically highlight relevant events. The user interface will allow the analyst to create his/her own cases in a user-friendly graphical way. Additionally, the analyst will be able to define his/her entities and events, visualize causal, temporal and spatial sequences and test or refine different hypotheses.
• A framework for evaluation through simulated procedures in order to have quantitative measures of the system results.
• A series of training sessions to analysts and operators to make them capable of using it efficiently and with ease.
All the above will be realised following a privacy-by-design approach to ensure privacy protection through built-in, rather than added-up, legal and ethical compliance mechanisms. It is worth mentioning that Legal, Ethical and Privacy (LEP) negotiation will be provided both per module and in a system level.
Project Context and Objectives:
The main goal of LASIE is to design and implement an open and extendable framework that will significantly facilitate the investigation process of the analysts, providing them automatic analysis services of vast amounts of heterogeneous forensic data. This framework guides the investigation procedure through the incorporation of recommender services and interacts with users though efficient and user-friendly graphical user interfaces. LASIE will be performed under the condition that all legal and ethical restrictions are satisfied and the computed evidence data can be presented in European courts of law.
The main LASIE objectives can be broken down into the following 11 objectives:
1) Tools for supporting automated processing of very large amounts of data (WP4): An efficient way to address the processing of large datasets is to reduce the search space by applying advanced media analysis techniques, fusion of information and filtering based on time/space/event/pattern dimensions of the information. LASIE will develop and test advanced media analysis tools for the different types of content (text, audio, images, video, social and biometric data, etc.). These tools will extract relevant low-level and high-level features to be used in the next steps of the investigation. The objective of the 3rd reporting period is the finalisation of the forensic data processing modules that were developed during the 2nd reporting period (further improvements based on the comments of the 2nd review). The goal is to prepare the final versions for integration in LASIE platform.
2) Domain knowledge acquisition and modelling (WP5): In order to enable an efficient investigation, information gathered from multiple and heterogeneous sources has to be transformed into usable knowledge. Such a transformation will involve the use of analysis tools and modules to extract mid-level semantics from the forensic data. Upon the extraction, with the help of generic intelligence knowledge representation schemas, appropriate knowledge structures will be developed. In LASIE a comprehensive knowledge representation framework for modelling, indexing, classification and retrieval of data for forensic applications will be developed. The knowledge representation framework will integrate diverse sources of information including geospatial grid representation, temporal semantics, event representation and object annotation through a multimedia ontology. During the 3rd repotting period, the objective is the finalisation of the LASIE knowledge representation framework and tools for event-based modelling and extracting data patterns and anomalies.
3) Development of an Intelligent Evidence-based Search Engine (WP6): The search engine will enable search and retrieval in both the existing forensic databases and the various captured forensic data. The searchable data types comprise text (in the form of documents, email, internet history, calligraphic, etc.), images (faces, tattoo, plates, etc.), video (stored in local disks, surveillance, etc.), audio (speech, emotions, etc.), social and biometric data, i.e. all types of forensic data. Due to the high heterogeneity of these data types, several search modules will be implemented, each one adapted to the specific technical constraints of each type. The individual search modules will exploit technologies for distributed processing and large-scale indexing to enable fast and efficient search even to vast repositories of forensic content. Indexing will be performed on the metadata extracted with the processing tools. The objective of the 3rd reporting period is to implement the LASIE multimodal search engine that will integrate individual search modules, the final version of the complex query formulation and relevance feedback. Moreover, the finalization of the crowdsourcing tool for super recognisers is one of the main objectives within the WP6.
4) Provision of guidance to analysts through inference and recommendation functionalities (WP7): The Recommendation and Inference modules will be designed and developed within LASIE to achieve the following objectives:
• To continuously provide possible investigation directions (e.g. through the presentation of information relevant to the already provided evidence data) to the analyst, who will eventually decide whether to pursue them or not.
• To perform automated inference based on the provided evidence, in order to detect and resolve possible inconsistencies or to reduce uncertainty associated with the estimated analysis results.
• To highlight important information, suggest relevant information sources and supply additional information regarding a particular entity, based on its relations to other entities.
The objectives of the 3rd reporting period are:
• To finalise the development of the recommendation module and integrate it in LASIE framework.
• To finalise the development of the inference module and integrate it in LASIE framework.
• To update the Evidence graph (EG), integrating new developments of WP4 and WP6 modules.
5) User Interface that enables Presentation of Highlights and Refinement of Investigation Process (WP8): LASIE will provide visualization of the results in order to present relevant highlights addressing the who, where, when, what and why, and give the analyst the capability to refine his/her search queries and hypotheses. LASIE will develop a user interface specifically designed for forensic purposes. The user interface will provide modelling and simulation in order to test operational procedures and techniques through visual analytics and crime scene reconstruction modules. Special focus will be given on its design in order to produce a user-friendly GUI that can be easily manipulated even by non-IT expert police officers. The objective of this period is the development of updated LASIE User Interface, by integrating query formulation, visual analytics and crime scene reconstruction, taking also into account feedback/recommendations from the previous version.
6) Legal and Ethical Compliance through Privacy by Design (WP2): The LASIE framework will apply a Privacy-by-Design methodology, providing built-in privacy protection mechanisms rather than added-up security. The objective of the 3rd reporting period is to follow the methodology to encure legal and ethical compliance during the design and development of all LASIE modules, as well as development of the ethics and digital evidence (EDE) module.
7) Continuous integration and testing (WP9): Following the system requirements and architecture (WP3), the objective of the 3rd reporting period is to implement the final version of the LASIE platform by integrating the final versions of all LASIE modules. Moreover, the system testing and validation is planned for this period.
8) Pilots and user evaluation (WP10): An objective of the 3rd reporting period is also to prepare the final arrangements of the LASIE pilots at MET and ADM premises, to run the pilots using the final integrated version of the LASIE platform and prepare a framework for user evaluation thourgh questionnaires appropriately designed to evaluate the user requirements.
9) Project dissemination and training (WP11, WP12): LASIE dissemination aims to maximise social, scientific and industrial visibility of the developments, findings and outcomes of the project. Apart from disseminating LASIE through industrial fairs, workshops/conferences, paper and internet media, publications in international journals and conferences, it is important from the very beginning of the project to:
• setup and maintain the project website
• prepare LASIE Newsletter (updated every six months)
• setup a LASIE Advisory Board, composed of end-users and security authorities with the aim to support the Consortium in the definition of operational requirement, scenario prioritization, testing, validation and assessing of the crowd management solutions and methods adopted and developed in the project.
Regarding training activities, an objective of this reporting period is to organize the third LASIE training workshop and prepare the supporting training material.
10) Exploitation of project results (WP11): The objective of the 3rd reporting period is to finalise the market analysis and exploitation plan that were initialised during the previous reporting period. The exploitation plan shall include the tools that will be available by the end of the project.
11) Contribution to Standards (WP11): The objective of this period is to proceed to the schedule structured within the previous reporting period and finalise the standardization activities based on this schedule. The activities should be aligned to the plan of the research conducted in WP4-8.
Project Results:
A summary of the work and main results achieved during the third reporting period are summarized below:
1) Tools for supporting automated processing of very large amounts of data (WP4): All planned modules for processing of forensic data were finalized and integrated during the third reporting period. More specifically, in text processing, new services were implemented for dealing with Jargon and specific terms used mainly in the Social Media contents; in image processing, new modules were implemented for logo detection, face detection as well as the EXIF module, following the recommendations of the second review; regarding video processing, the modules for crowd violence, group detection and people tracking were finalized and demonstrated in MET data. Morever, all social media crawlers have been finalized and provided for integration as services outputting a valid JSON object containing the retrieved content. All related updates have been reported in reported in Deliverable D4.7 Final version of LASIE forensic data processing tools (M24). D4.7 includes further details on minimum resolution for face and plate detection.
2) Domain knowledge acquisition and modelling (WP5): The most important result in this area has been to move the knowledge acquisition from surface text to deep semantics, which in many cases is very different from the surface appearance. This novel step is crucial in order to be able to reason correctly about the data. The LASIE Knowledge Representation Repository was improved, in order to achieve an ontological model to integrate all heterogeneous data. Moreover, a refinement of existing supportive ontologies (Context, Category, Entity, etc) based on UC2 proposed scenario took place. The visualization toolkit was developed by SEN in order to visualise immediately the most important aspects of the story, namely its causal and temporal sequence, and in its ability to restrict or expand the views of events and entities. Finally, the knowledge structure for LASIE event-based knowledge representation was improved, achieving a modular ontological model for LASIE forensic domain. An Ontology Manager was defined and developed, which provides the main functionalities to manage ontologies through API. The definition of recurrent knowledge graphs represents common complex information to store into knowledge base, to retrieve information in a more easy and complete way. The work in all the above activities is reported in Deliverables D5.3 Knowledge Representation Repository and supporting tools (Update of D5.2) and D5.4 Report on Event-based Knowledge Modelling (M34).
3) Development of an Intelligent Evidence-based Search Engine (WP6): One of the most significant results, during the 3rd reporting period, is the finalization of the LASIE multimodal search engine, which integrates the latest updates in individual search (text, image, audio, video) modules, the Complex Query Formulation (CQF) and the Relevance Feedback (RF). The achievements are reported in Deliverable D6.3 LASIE multimodal search engine (2nd iteration) (M34). Moreover, the TooManyEyes Crowdsourcing tool for Super-recognizers was finalized after collaboration between CERTH and UoG. The outcome was a tool of high quality, thus, an initial plan for commercial exploitation of the tool has been prepared. The achievements are reported in Deliverable D6.4 Mechanisms for relevance feedback and human-driven search refinement (M34).
4) Provision of guidance to analysts through inference and recommendation functionalities (WP7): During the 3rd reporting period, the achieved results in objective 6 are listed below:
• The Evidence Graph (EG) has been updated based on the feedback of the previous reporting period, in order to integrate new features.
• The LASIE recommendation module has been finalised and integrated in LASIE framework, supporting all three Use Cases. This work has been reported in Deliverable D7.4 Evidence Recommendation module (2nd iteration) (MS38).
• The LASIE inference module has been finalised and integrated in LASIE framework. A mechanism to retrieve relevant knowledge sub-graph. Moreover, a semantic approach, based on a reasoner, ontologies and rules, to create hypothesis for the proposed scenarios was defined and developed. This work has been reported in Deliverable D7.3 LASIE Inference Framework (2nd iteration) (MS33).
5) User Interface that enables Presentation of Highlights and Refinement of Investigation Process (WP8): An updated version of the LASIE User Interface was delivered within the 3rd reporting period, to support all three Use Cases in the pilot activities and the third LASIE Workshop. All recommendations from the 2nd review and user feedback were taken into account to improve the UI in several aspects. All modules, i.e. the Query Formulator (QF), the Visual Analytics and the Crime Scene Reconstrution (CSR) were integrated. The approach for assessment of the User Experience has been followed to ensure that the UI has been improved according to the recommendations. All the above work has been reported in Deliverable D8.3 The LASIE User Interface (2nd iteration). The work in Visual Analytics module and Crime Scene Reconstruction module has been included in D8.1 Video and Visual Analytics and D8.4 Interface for crime scene reconstruction, respectively.
6) Legal and Ethical Compliance through Privacy by Design (WP2): During the third reporting period, the information sharing agreements were finalised and DPAs were notified, where applicable. Moreover, the implementation of the ethics and digital evidence (EDE) module was finalised. The module ensures that both the requirements concerning privacy, personal data protection, ethics of surveillance and laws on digital evidence, as well as the principles of Responsible Research and Innovation (RRI), are adequately addressed in the LASIE prototype. Deliverables D2.3 Monitoring reports on emerging ethical challenges in the developing and implementing the system (Update of D2.2) (M42) and D2.5 Data control report (update) (M42) have been finalised and submitted by M42. Deliverables are updating the recommendations on how to take into consideration the societal issues raised by LASIE (data protection, ethics, privacy and so on), by analysing the technological landscape and change, legislative and societal change.
7) Continuous integration and testing (WP9): Most important results achieved in last months that impact on several aspects of the platform, are:
• Quality of data was improved through data filters that works transparently for the user, especially for crawlers and NLP modules;
• Modularization of the interpreter that processes the scripts;
• Implementation of new data fusion services that apply domain rules (based on ontology and data fusion strategies) that improves the quality of the knowledge base and allow to merge data that refers to the same entities but arrives fragmented and from several sources (e.g. data that arrives from two modules but refers to same event);
• Implementation of consistency rules and services that manage better relations between resources and entities (e.g. event creation starting from a video resource);
• Implementation of high level integration services (described in the D9.3) that simplify many operations performed by the GUI and that are based on the low level integration services (described in the D9.2);
• Simplification of the script language used to define the logic that describes how orchestration is done;
• Fully integration of knowledge patterns in order to ensure an high grade of consistency (an important issue in domain driven approach, see D5.4);
• Integration of ontological validation rule in all integration services and in general all new features provided by the knowledge management framework;
• Refinement of workflow with changes to existing integration services in order to meet some reviewers and users suggestions and to simplify the orchestration pipelines;
• Re-writing of some low level services in order to improve performances;
• Development of services for system monitoring and user action logs in order to improve the governance layer and meet some reviewers suggestions;
• The chosen methodology of integration of information extraction modules has demonstrated to respond well to the needs of the project;
• A new management of Hypothesis (WP7) at business logic was done.
Integration Framework also support GUI and integrate all module expected (as described by the WP3) are working. Prototype integration is actually completed. This work has been reported in Deliverable D9.3 Final Integrated LASIE framework (M42). According to the objectives of T9.4 NP worked together with ENG, CERTH, INNEN, UPM in order to elaborate a consistent testing and validation plan adapted to the identified needs of the overall LASIE framework. The validation plan was divided into two distinct processes: one focusing on validating each individual component against its requirements and one that was dedicated to the validation of the integrated LASIE platform. The validation and testing plan was guided by ISO 9126’s main principles for software quality assessment and its dimensions, subcategories and quality metrics were taken into account throughout the task activities. The first step for the individual validation of LASIE’s components was to determine the method by which each requirement would be validated. All consortium partners were involved in this process and a well-balanced mix of different validation methods has been chosen, based on requirements’ type and fit criterion. Finally, the technical partners of the consortium have led the technical validation activities and provided their findings for all the components that are involved in the LASIE’s Use Cases. For the validation of the integrated LASIE platform a set of quality metrics has been defined to monitor and assess the quality of the developed software based on ISO 9126 dimensions. The actual testing phase was carried out at the integrated LASIE prototype and targeted to confirm the quality of the software. LASIE platform was tested in an operational environment, and in selected test cases, thus, ensuring that the working blocks of the systems have been integrated successfully and work in conjunction. This work has been reported in Deliverable D9.4 LASIE technical validation (M42).
8) Pilots and user evaluation (WP10): The work in this WP for the 3rd reporting period includes the following:
• Successful realization of pilots in June 2017 in London (MET premises) and Madrid (ADM premises).
• Preparation of dedicated questionnaires and training materials for assessment and evaluations;
• Execution of evaluation session with internal and external end-users;
• Analysis of gathered feedback and production of D10.3.
The work has been reported in D9.2 Demonstration portal (M36) and D9.3 LASIE framework evaluation (M42).
9) Project dissemination and training (WP11, WP12): During the 3rd reporting period, the number of Advisory Board members has grown with a new expert (Mr Gianfranco Todesco, chief Commissioner of the Digital Investigations team at the Turin Local Police):
• Ten members from seven different countries (ES, GR, LT, PT, UK, FR, IT)
• Three members from international bodies
The consortium disseminated the project through discussions and presentations with technological partners and end users in three industrial events of the security market. The project was also disseminated within scientific conferences through high ranking at the DCASE conference challenge, a booth but also with the implementation of special sessions in ICIP and ICDP 2016 international events. Regarding scientific communications, two new journal papers and four new conference papers have been accepted for publication. Also, 7 talks/keynotes in universities and international conferences can be added to the dissemination listing. The project also organized two lunch seminars at PRIO premises and co-organized two workshops in London in the past months.
The work has been included in D11.3 – Report on Dissemination Activities (Update) (M42).
Two training sessions for end-users were organised in Madrid (21st June 2017) and London (30th June 2017) as well as one final training workshop in Turin (28th September 2017). In particular, the final workshop was held in Turin at the Turin University with the intermediation of the Advisory Board (in particular the Turin Local Police). The training was focused on practical individual sessions, where end users learned how to interact with the system. In Turin, the practical session was preceded by the explanation of the main features and on the usage of the system by end users. A roundtable of end users, researchers, Advisory Board members and representatives from sister projects concluded the workshop. The work has been included in D12.5 3rd Workshop Report (M42).
10) Exploitation of project results (WP11): With respect to exploitation, LASIE applied and approved to benefit from the Common Exploitation Booster support services offered by the European Commission H2020 Common Support Centre. EC has assigned to LASIE a coaching expert: Mr. CHICHOCKI Tomasz from META Group. Then, an Exploitation Strategy Seminar was organised in Oslo on June 13th, where all LASIE partners participated. Several results of the LASIE project were assessed and initial exploitation plans have been drafted by the involved partners. The work has been included in Deliverable D11.6 - Market analysis & Final Exploitation Plans (M42).
11) Contribution to Standards (WP11): In the current reporting period, VML participated in two MPEG meetings held at Chengdu, October 2016 and Geneva, January 2017 and contributed towards the development of Media Orchestration standard. The application of Audio-Visual patterns through the extraction of DROP features was included in the standards specification as normative reference. The bit stream representation for DROP features was standardised using the MPEG-B specification and in the standard, has been referenced as Region-of-Interest (ROI). The representation of the ROI uses the MPEG-B part 10 Amendment 1 (via ISO/IEC 23001-10) uses ISOBMFF file format for pointers to 2DCartesianCoordinateSamples, the 2DCartesianCoordinateSampleEntry, their timing and their relationship to video samples in the ISOBMFF file. Clause 6.2.2.1 specifies the syntax of a 2DCartesianCoordinateSample, which are 9 bytes in byte (8-bit) alignment. Zero or more of these 9-byte 2DCartesianCoordinateSample are contiguously contained inside an 'mdat' box. Clause 6.2.1.1 specifies the syntax of a 2DCartesianCoordinateSampleEntry, which is a MetadataSampleEntry of the type '2dcc'. Zero or one 2DCartesianCoordinateSampleEntry is contained inside a 'stsd' box.
The following input contributions were made on behalf of LASIE project to the MPEG MORE ad-hoc group. VML also developed a visualisation of the DROP based orchestration which was also presented in the LASIE review meeting. In the forthcoming period, VML will continue to contribute towards the reference software implementation.
Input contributions:
• Krishna Chandramouli, Panos Kudumakis, Ebroul Izquierdo, "LASIE Dataset for Media Orchestration”, ISO/IEC JTC1/SC29/WG11 (M39214), Oct 2016 (15/10/2016)
• Krishna Chandramouli, Panos Kudumakis, Ebroul Izquierdo, "Representation of Orchestration Data based on visual patterns”, ISO/IEC JTC1/SC29/WG11 (M40118-v4), Jan 2017 (14/01/2017)
The work has been reported in D11.7 - Report on standardisation efforts (M42).
Potential Impact:
LASIE impact on the “cost of the crime” and resolution of crimes:
The economic and social “cost of the crime” is an essential measure of the impact of the crime on the society. The cost of the crime gives us a way of measuring the impact of policies aimed at reducing crime and its consequences. Thus, the cost of crime has become an increasingly important tool for decision-makers concerned with crime’s impact on society. It helps make explicit judgments about the relative merits of alternative policies and programs, which are already implicit in decision-making about how to allocate resources to tackle crime – both overall and between different types of crime. According to the Home Office research report of October 2013, the social and economic costs of organised crime amount to many billions of pounds. Drugs supply (£10.7 billion), organised fraud (£8.9 billion) and organised immigration crime types (£1.0 billion) have major impacts on the UK, and other less visible crimes also cause substantial harm. This report outlines evidence on organised acquisitive crime types; organised child sexual exploitation; counterfeit currency; drugs supply; organised environmental crime; firearms; organised fraud; organised immigration crime; organised intellectual property crime; and organised wildlife crime, which all cause damage to the UK.
Moreover, crime volume in LASIE’s partners’ countries of origin (Germany, England, Wales, France, Italy, Belgium, Spain, Greece, Portugal, Norway, and Ireland) has reached 11.66 million police registered cases.
As far as the distribution of crime types in the LASIE’s partners’ countries of origin, it is evident that theft and burglary direct and indirect crimes occupy 80% of crime volumes, then followed by 10% assault crimes.
Moreover, if we take into account the total cost of the crime to the society, this can be much higher than the cost of the damages or the lost/damaged property. The property loss is only 10%, however the emotional and physical impact on victims is calculated to 50%, while response to crime by the criminal justice system constitutes 20% of the total cost of crime.
What is even more important is that only 1/4 of the crime is known to the police, less than 1/10 is cleared and only 1% is finally sanctioned (jail, fine, alternative punishment). Only in 2011, a third of crimes went unsolved, as police admitted there is little chance of a positive outcome.
Crime reduction and criminal policy is making progress but still, in the case of civil service tribunal, the number of pending case remains quite high, while in case of General Courts Tribunal the number is even increasing.
LASIE aims to significantly facilitate the investigation process of the analysts, by providing an automated initial analysis of the vast amounts of heterogeneous forensic data, guiding the investigation procedure through the incorporation of a recommendation functionality. This approach will not only increase the number of resolved cases and therefore reduce indirect cost of the crime, but will also lower the cost of investigation.
LASIE impact on legal and ethical issues:
Legal and ethical issues concern any investigation system since they are dealing with personal data. The principle is that "a person is innocent until proven otherwise". Yet, in many investigations, involving persons that are finally proven to have nothing to do with the case, may significantly harm them, their personality, their career, even their relatives.
LASIE carefully considers the legal, ethical and privacy issues. Experts within the project have studied these issues in D2.1 “Legal and ethical guidelines on data collection”. Moreover, the LASIE framework respects privacy and ethical issues by design. Such a result will increase public acceptance of next generation security system.
LASIE impact on increased (feeling of) security:
Fear of crime has become an important issue of public concern: a problem which detracts from the quality of life, and which adversely affects social and economic well-being. While the fear of crime expressed by some citizens is well-founded, other individuals are at less personal risk than they might believe. Their fear, however, is no less real. The levels of fear of crime in European neighbourhoods grow to reach 40% of the population in certain countries.
We expect that the increased effectiveness of LASIE as an investigation system will have a positive impact not only on the actual level of security but it will also positively affect the psychological feeling of security, resulting in a positive effect in every day quality of life especially in the social environment.
LASIE impact on increased trust in Police:
The trust in Police forces is also an important issue. A high level of trust is necessary in any democracy to ensure Police credibility and authority. It appears in different surveys that the citizens’ satisfaction rate of Police performance remains rather low. A recent survey in England and Wales reports that only two thirds of people think the Police is doing a good job. The satisfaction rate falls to only 39% in rural areas. Some other consequences may be a certain feeling of impunity to the criminals. By increasing the number of solved crimes and the delay in which they are solved, LASIE will indirectly participate in the increase of trust in Police forces.
LASIE impact on economic growth and job creation:
LASIE develops original state of the art know-how (and later on skills) in a specific area of investigation devoted to security systems applications. This area has significant economic potential itself. Yet, the LASIE results are achieved in the broader area of security and data analysis processing, which is a powerful driver of economic growth (and employment) in today knowledge-based economy. In short, LASIE will help to create opportunities in the specific area of investigation systems but a larger fall-out of some of its results can be envisaged in all areas where data processing can be harnessed to support security and innovation.
The market sector related to LASIE is the one related to forensic sector, in particular law enforcement investigators/analysts dealing with the analysis of different sources of forensic data coming from surveillance systems, voice recording devices, computer/mobile-phone network monitoring platforms, the networked social services. Such market is growing at a very high pace in recent years.
Modern societies are confronted with an increasing number of illegal events and crimes. Such extreme events are marked by threats to societies’ values and/or life-sustaining functions and create an urgent need to respond to them as quick as possible. In case of crimes, it is the responsibility of forensic sector to manage the investigation process in order to save lives and restore a sense of order. One of the constant challenges encountered by forensic data is the search in extremely large collections of heterogeneous forensic data, i.e. the information that is required for making high-quality decisions under pressure. Without accurate analysis, analysts/investigators find it hard to make fast and correct decisions. In fact, the absence of reliable evidences (combined with the flood of unsubstantiated reports and rumours) could jeopardize a fast resolution of the crimes. To solve such problems, LASIE develops appropriate (semi-) automatic tools for facilitating the analysts in their investigation activities. Such added value of LASIE will represent its unique selling proposition in the forensic sectors.
Due to a unique confluence of technological change and social demands, the forensics business is one of the most dynamic sectors of the modern economy. The capability of forensic techniques has grown, costs have dropped and hence decision makers have promoted forensics to a generally receptive public. The demand for products and services is therefore generally on an upswing. Based on research and survey, it is projected that total combined spending on forensic investigation by public and private sectors will increase year after year. Technology advancements in forensics have been of considerable import in recent years. BCC estimates that total sales of forensic products and services were nearly $10.1 billion in 2010 and that sales will grow at a 9.4% compound annual growth rate (CAGR) through 2016 to more than $17.2 billion. Other studies project that the US market for Forensic Technologies & Services will even register more than $20 billion in annual revenue by 2020.
LASIE, therefore, will provide its unique platform towards a market that is expected to increase its spending constantly, in an overall trend that is making security one of the fastest growing and challenging emerging business environment.
Key points for the commercial exploitation strategy has been determined making the basis for the route-to-market.
LASIE partners estimate that most of the expected exploitable results can be ready for commercial exploitation between 9 to 18 months after the project end.
At this stage of project development, the exploitation strategy planning foresees the following principles and route-to-market:
• The business models explored are licensing and SaaS, but other models will be taken into consideration according to the needs of different potential users and based on innovative adoption models. The starting assumptions are that the outcome of LASIE should be provided as a Software as a Service (SaaS), allowing end-users to sell the service to potential clients, rather than as a stand-alone software or perhaps as Cloud-solution for ultimate scalability and agility. We believe the real value of the test lies in the service provided by professionals who can use potential innovation on day-to-day activities which will spread information about LASIE in the forensics and security sectors.
• The Consortium is evaluating two main joint exploitation routes: (1) a joint-company is established amongst the LASIE partners, targeting the European market in collaboration, or (2) provide partners/end-users access to use the service, with centralized hosting, in effort to aggregate results from across the European market, making it possible to conduct in-depth research of human behaviour in advanced forensic and investigation activities, which may prove more valuable than the platform itself.
• LASIE platform can be exploited under three main packaging combinations: (1) light, (2) intermediate, (3) complete package. different packages can include different modules according to the real needs of the final users. The cost for each package will vary based on available modules.
• Importance to provide a customizable commercial solution able to guarantee different levels of needs for every customer, no matter which sector the target user is operating in and its company size;
• Starting from markets in Belgium, Germany, Ireland, Greece, France, Portugal, UK, Spain, Ireland and Italy, then all EU markets will be targeted in order to fulfil the business scenarios and expand the market presence of partners across Europe;
• A deep and detailed refining of the marketing and commercialisation plan need to be delivered later by specialized commercial Business Units containing all the activities to be carried out, containing strategies for acquiring customers and detailed services to be provided;
• Project partners aim to conclude some preliminary contracts with large and/or intermediate organisations, in the months right after the end of the project, in order to start the commercialisation phase as soon as possible or to find new funding opportunities for further product development. To this end, partners will leverage on their existing networks, counting mainly on public bodies and big organizations;
• The overall marketing strategies will be defined internally by skilled marketing and sales units and implemented by all companies by their business units. In particular, industrial companies through LASIE will empower their current product portfolio and will be able to provide it to their existing customers. Moreover, they are confident to boost their business opportunities also by selling to new customers also products and technologies already present in their current portfolio.
• A specialized long-term learning programme for the employees will be developed, on a periodical basis, addressed to the vulnerable groups. Furthermore, upon necessities, this programme will allow instant actions.
For all Key Exploitable Results (KERs) some common exploitation and commercialization issues arise:
The primary market of the consortium is a challenging one. Apart from the in-depth market and competition analysis the interested partners should better understand the decision-making process specific to this market and prepare the message for the key stakeholders. Alternatively, some of the results could be successfully used outside forensic and security field, such as broadcasting and entertainment industry which is less regulated and more market oriented. The consortium should take the advantage of Advisory Board members in finding the potential new customers.
The successful market introduction of project’s KERs will require significant financial investment. For relatively low TRL results the new Horizon2020 project could be a proper path for the development. The industrial partners may benefit form Horizon2020 SME Instrument and support of Enterprise Europe Network. Partners looking for private funding may use the European Investment Project Portal (EIPP) to look for the potential investors.
List of Websites:
http://www.lasie-project.eu/