Final Report Summary - VIRTUOSO (Versatile InfoRmation Toolkit for end-Users oriented Open Sources explOitation)
Executive Summary:
VIRTUOSO provides a technical framework for the integration of tools for collection, processing, analysis and communication of open source information. "Plug and play" functionalities that improve the ability of border control, security and law enforcement professionals to use data from across the source / format spectrum in support of the decision making process are enabled by this middleware framework. As a proof of concept and to highlight the efficiency of this open-source code framework, a prototype has been built and demonstrated using operational scenarios.
The project, developed by a consortium of 16 European organisations, is co-funded under the FP7 programme of the EU and will comply with legal considerations, enforcing the principles of privacy
and data protection to ensure the interests of citizens within the European Union. The main functional components of the VIRTUOSO framework are:
- Information Gathering components: Collect the information coming from multiple supports (disks, intranet, the Internet...) multiple formats (paper, digital, analogical...) and of multiple natures
(textual, audio and visual (image and video)), and then convert data into standard format. For instance paper documents can be scanned and translated via OCR API into XML output format, in
the same way the speech audio documents are converted, using speech to text API, into XML format.
- Information Processing and Structuring components: Structuring Open Source Information to Support Intelligence Analysis. It deals with information reduction and structured based on event and
information extraction. It comprises information filtering and categorization, Information Retrieval, Entity and event extraction, Information summarization.
- Knowledge Management components: These services deal with the representation and the exploitation of the knowledge built from the extracted information; A VIRTUOSO Knowledge Base
(VKB) has also been developed in order to provide a uniform data model satisfying all the needs for representing and using operational and conceptual knowledge
- Decision support components: Provide a substantial aid to the decision makers by integrating various sources of information and by developing tools to exploit the knowledge stored in the VKB and components for Situation Assessment, Scenario Building and Misinformation detection.
- Visualization components: Provide an intelligent access to visualize relevant knowledge, and aid the process of structuring the work and the decisions upon the extracted information.
- Evaluation components: Evaluate atomic services and processing chains to compare the capability and performance of different services or chain configurations in specific task such as: named entity, extraction, speech to text, etc.
To deal with the integration of these different components, the proposed framework is composed of 4 layers:
- Processing framework: collect, analysis, indexing, etc.
- Decision support framework: knowledge management, reasoning, etc.
- Visualization framework: unified HMIs, interoperability mechanisms, etc.
- Evaluation framework: mechanisms for the evaluation of atomic services and processing chains
Project Context and Objectives:
Open Source Intelligence (OSINT) is an information processing discipline that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence.
The previous trend has been for intelligence agencies to primarily focus on obtaining information that is not available to ordinary citizens, though networks of agents, or the monitoring of electronic communications. OSINT is to play a pivotal role in today’s information society with the ever-growing amount of information publicly available.
Benefits
• Speed and coverage
As Stephen C. Mercato wrote in “Reexamining the distinction between open source information and secrets”, ‘There are more bloggers, journalists, pundits, television reporters, and think tankers in the words than there are case officers” .
• Quantity
OSINT is drawn from publicly available material including, the Internet, traditional mass media ‘television radio, newspapers, magazines), specialized publications (scientific and specialized journals; conferences proceedings, think thank studies), pictures and photos, geospatial information (maps, commercial imagery).
• Quality
“The OSINT target is immense. Engineers attend conferences; scientists publish scholarly articles; manufacturers build production lines; bureaucrats issue guidelines; and traders print brochures for prospective clients. Many paper trails wind around the world beyond whatever may surface in the media.”
• Ease of use & extended monitoring
• Cost & Risk management
Challenges
The key role of Internet as a vehicle for spreading information is undeniable. One of the main challenges is to manage the enormous volume and flow of information. Traditional media also use the same information and validation channels as their audience. This increases the possibility to influence because transmitters and receivers tend to become intertwined.
Therefore, the second challenge is for intelligence analysts to identify all possible actionable intelligence and analyze it. If open source information is not classified, it does not mean however that it is easy to find or even understand and interpret.
Finally, in OSINT, the ability of the intelligence analyst to analyze is at the heart of the discipline. The difference between the two classifications (Open vs Classified / Secret) depends as much on the ability to analyze as on the level of the information received. However, it mostly depends on the ability of the user to disseminate, re-arrange, and interpret information.
The main functional components of the VIRTUOSO framework are:
- Information Gathering components: Collect the information coming from multiple supports (disks, intranet, the Internet...) multiple formats (paper, digital, analogical...) and of multiple natures
(textual, audio and visual (image and video)), and then convert data into standard format. For instance paper documents can be scanned and translated via OCR API into XML output format, in
the same way the speech audio documents are converted, using speech to text API, into XML format.
- Information Processing and Structuring components: Structuring Open Source Information to Support Intelligence Analysis. It deals with information reduction and structured based on event and
information extraction. It comprises information filtering and categorization, Information Retrieval, Entity and event extraction, Information summarization.
- Knowledge Management components: These services deal with the representation and the exploitation of the knowledge built from the extracted information; A VIRTUOSO Knowledge Base
(VKB) has also been developed in order to provide a uniform data model satisfying all the needs for representing and using operational and conceptual knowledge
- Decision support components: Provide a substantial aid to the decision makers by integrating various sources of information and by developing tools to exploit the knowledge stored in the VKB and components for Situation Assessment, Scenario Building and Misinformation detection.
- Visualization components: Provide an intelligent access to visualize relevant knowledge, and aid the process of structuring the work and the decisions upon the extracted information.
- Evaluation components: Evaluate atomic services and processing chains to compare the capability and performance of different services or chain configurations in specific task such as: named entity, extraction, speech to text, etc.
To deal with the integration of these different components, the proposed framework is composed of 4 layers:
- Processing framework: collect, analysis, indexing, etc.
- Decision support framework: knowledge management, reasoning, etc.
- Visualization framework: unified HMIs, interoperability mechanisms, etc.
- Evaluation framework: mechanisms for the evaluation of atomic services and processing chains
Project Results:
Definition of the core platform
The Virtuoso core platform consists of several open source based components:
The main contribution in the platform is Weblab, the other part is ING. All projects members worked in synergy to embed all parts of the platform in one. As a result, Weblab C++, evaluation framework and presentation framework are integrated into Weblab repositeries as part of Weblab. In this document, we will refer the the platform as the association of Weblab and ING.
Weblab: an open source platform for supporting SoA applications
Virtuoso platform is composed of four coherent groups of components, named frameworks. These frameworks are detailed below.
The processing orchestration framework: WebLab is the chosen platform for the VIRTUOSO platform and used as basis for the VIRTUOSO processing framework. WebLab provides a set of tools and methods that aim at building information systems for intelligence applications in the economy, strategy and military domains.
The evaluation framework: Based on the WebLab platform, it allows evaluating WebLab processing components in terms of the quality of the underlying algorithm. For example, a named entities extraction service will be evaluated in terms of precision and recall of the extracted entities. The evaluation services will wrap evaluation campaigns based tools inside a WebLab interface.
The decision support framework: The ING Semantic Middleware, ING for short, was chosen to provide the basis for the VIRTUOSO decision support framework in the VIRTUOSO platform. It provides functionalities to interact with the KB. For the VIRTUOSO demonstrator, the KB solution will be ITM (Intelligent Topic Manager), provided by MONDECA. The ING core architecture implementation already provides a generic driver for the ITM access, but for the VIRTUOSO demonstrator, it would be required to develop a specific VIRTUOSO meta-model ITM driver.
The unified presentation framework: it ensures the standardisation of the user interfaces of the application upon the platform. This covers three different aspects: graphical aspects, integration methodology and standardization of communication between user interface fragments. As part of the integration in the global architecture the presentation framework relies on a web interface hosted on a web portal and each interface component will be integrated using common technologies.
WebLab: an open source platform for supporting OSINT applications
The WebLab framework is an open source technical baseline acting as a runtime environment for unstructured information processing services. It provides a set of tools and methods that provide easy and flexible building of media-mining applications (OSINT, business intelligence, etc). Typically, WebLab handles multimedia and unstructured data (text, image, audio and video) through heterogeneous business components providing services.
The WebLab platform relies on a SOA (Service Oriented Architectures) as the core paradigm for the design and integration of components. The high-level functions, offered to users through applications, are achieved by chaining together services and calling them in the right sequence. As a consequence, the service definition and conception is a key feature in the platform.
The WebLab technical Architecture can be seen as a multi-layer architecture.
Having a top down view, the WebLab architecture consists on the following layers:
• The access layer provides (i) graphical user interfaces (GUIs) that enable “final” users to access services in a hidden way, (ii) a monitoring application that enables "admin" users to control and check the execution of the processing chains and(iii) an administration application that includes tools supporting business workflow deployment and configuration and enabling “architect" users to define service chains.
• The process layer or orchestration layer consists in an orchestrator that acts as an intermediary between services in order to perform a business process.
• The service bus layer is in charge of communication and distribution of messages between services in accordance to the request expressed by the orchestrator. It allows abstraction of services localization and implementation as well as abstraction of communication protocol.
• The service layer contains standardized services that are developed upon the WebLab Data Exchange Model and implement WebLab Generic Interfaces. These services act as connectors to technical & business components.
• The component layer is where developed components (that are hidden behind the standardized services) are located.
• The data layer contains (a) repositories used by the platform for the storage of original content of documents and their description according to the WebLab Model (after processing), (b) indexes and (c) knowledge bases.
Each component integrated in the WebLab platform is developed upon the WebLab Data Exchange Model that will describe the resources used within a WebLab application, and implements one (or several) WebLab Generic interface(s) that defines their input and output in terms of WebLab data exchange model objects.
WebLab has a number of significant benefits:
o It allows fast creation of processing chains to meet any user needs.
o It provides a flexible solution where components can be interchangeable according to the requirements of the specific tasks.
o It enhances interoperability and intercommunication of heterogeneous systems, services and processes.
o Provides the use of standard security protocols to allow protection of confidentiality and integrity of information across the system.
o It relies on a scalable architecture allowing the integration of mass data analysis components.
o It uses standards for media-mining and data fusion (XML, OWL, OGC, RDF, SOA, etc.).
ING: a decision support middleware
ING Semantic Infrastructure is an Open Source environment (LGPL2 licence) developed by Thales Communications and Security (TCS). ING Semantic Infrastructure is the heart of the Decision Support part of VIRTUOSO, as it is the integration platform used by user in the loop VIRTUOSO applications working on the Knowledge Base. ING was initiated during the design phase of French national security project (2006), and enhanced in the OSEMINTI (EDA 2007-2009) and VIRTUOSO (EU) R&D projects. It is now contributing to Thales solutions in the Security and Intelligence domains.
The purpose of ING Semantic Infrastructure is to facilitate the integration of Decision Support (user in the loop) components from various origins, based on the semantic technology. It brings the flexibility and expressivity of semantic technology to the world of decision making.
The semantic representation relies on the representation of knowledge as triples (subject – verb – object).
ING is compatible with the RDF and TOPIC MAPS semantic representation standards issued respectively by W3C and ISO.
Recognized for its flexibility, semantic representation is increasingly used in decision making systems. ING Semantic Infrastructure proposes a generic environment to develop Decision Making application based on semantic technology.
The main components of ING Abstract Semantic Repository are:
• ING abstract semantic repository which provides APIs (query, update, model query) to address any repository as a semantic repository.
• ING Extensions which offers common services dedicated to data exchanges (the “basket”), traceability, and generic interfaces to operational functionality such as similarity and data fusion.
• ING Connectors offers a connector to connect ING to Mondeca ITM repository, or for memory storage, soon a RDF connector will be released by Thales.
ING Semantic Infrastructure & basic applications
The major benefits brought by ING Semantic Infrastructure are:
• to make application components independent from the repository actually used,
• to provide model adaptability to the components,
• to introduce a specific meta-model, which differentiates operational links (shown to the users) from technical links (used only by applications and hidden from the users) avoids to overflow end-users with useless information.
By its open architecture, ING Semantic Infrastructure makes the integration of new Decision Support applicative modules easy.
A set of decision support applications (entity editor, model editor, collective fusion) developed by Thales on top of ING Semantic Infrastructure are part of the ING open source release. In addition, the visualisation modules (Graph Viewer and Tabular viewer) developed by ATOS in VIRTUOSO are ING compatible and will soon be released as open source software.
Evaluation framework
An important part of an intelligence analyst's work revolves around using the right tools for the right task. As such, Virtuoso aims to respond to the need to have an architecture and software solution geared towards automated comparison of software offered by vendors. The project answered this need in two parts: benchmarking and evaluation.
Benchmarking
Benchmarking guidelines were established to rate the following technical elements: technical readiness, ease of integration, documentation availability, etc. The complete guidelines are part of Output 2.7 (Task 2.4) – “Benchmarking Guidelines”. These are aimed at technology providers, for them to be able to provide standardized readiness information to intelligence agencies.
Evaluation
Evaluation seeks to rate the quality of data processing components output. The method chosen by the Virtuoso project is to encourage the use of international evaluation campaigns in order to rate specific tasks. To this end, the consortium has developed specifications for the integration of any evaluation campaign into the Evaluation Framework using WebLab specifications.
Evaluation campaign
An evaluation campaign is made of a kit designed to judge the performance of a given type of tool (named entity extraction, translation, summarization, categorization, etc) over a specific set of data (news reports, scientific papers, tweets, etc).
Well-known evaluation campaigns include CONLL and MUC, for text-based information processing such as named entities extraction or translation, or TREC for text and multimedia information retrieval .
The following is typically contained in an evaluation campaign kit:
• Evaluation data set (or Test data). A set of documents to be run through the component to be evaluated.
• Ground truth (or Gold standard). The best possible expected outcome for the evaluation data set. Usually this is obtained through manual or semi-automatic annotation.
• Evaluation tool (or Scoring tool). A tool which compares the ground truth with the output of the component and computes a set of metrics. These metrics, defined for the purposes of the campaign, are the quantitative result of the evaluation. Each tool has its own requirements in terms of I/O.
• Training data. Some evaluation campaigns require the use of specific training data on the component before processing the evaluation data set. In these cases training data is included. However components which do not require training do not need to use this data.
An evaluation is run over a component through the following steps:
• (optional) The component to be evaluated is trained using the training data provided by the evaluation campaign.
• The component processes the evaluation data set. The outputs are converted into the input format expected by the evaluation tool.
• The evaluation tool processes the component output and compares it to the ground truth. Specific metrics are produced to quantify the quality of processing over the data.
Evaluation framework
The evaluation framework seeks to give the ability to integrate any evaluation campaign into WebLab in order to evaluate the various components used in OSINT tasks. The consortium demonstrated the feasibility of the evaluation framework using data from the CONLL 2003 evaluation campaign to evaluate named entity extraction components used in Virtuoso.
The specifications of the Evaluation Framework are part of Output 2.2 – “Overall Architecture design v1” and have been released as open source as online tutorials in order to facilitate integration.
A practice the Virtuoso project sought to encourage is for intelligence agencies to define their own evaluation campaigns (data sets, metrics). While recognized as a standard, international evaluation campaigns often have the disadvantages of
• being too generic,
• focusing on a specific type of media such as newspapers, tweets or scientific papers,
• answering the needs of a specific research agency (ex. DARPA).
Thus it may be more beneficial for intelligence agencies to build their own evaluation campaigns with the help of scientific experts.
Integration process
The proposed architecture is service oriented (SOA). Its aim is to avoid the dependency between the components and dependency to the platform. Each software component in VIRTUOSO is developed as an independent service both on the processing or on the user interface level. The basic idea is that each component consume and/or produce data using the common exchange model but that one-to-one communication between components should be avoided except data repository. Thus each component is independent from the real implementation and location of the component producing the data for them or consuming the data they produce. The same process is used for the user interface level.
Thanks to this architecture, the developer partners have done minor adaptation of their own component to be “VIRTUOSO ready”.
The major steps of this integration are:
1. Encapsulation and validation of technical components developed/provided in the wp-4-5-6-7 into web services w.r.t WP2 specifications. This work is done in each technical workpackge and relied on the integration factories developed in the WP8.
2. Deployment of services on internal servers or on CASSIDIAN's servers.
Test and robustification of each web services (to address concurrency requests and scalability for example).
3. Integration into the demonstrator of each web services (processing chain, interoperability with other services, etc.). This work is done in the WP8.
4. Test and validation of the integrated service. This work is done in WP8.
The WP2 has provided the technical specification which contain: a set of a generic interfaces and their IOPE(to be implemented by services), an exchange data model, an annotation model, etc. To do the step 1, the developer should select generic service interfaces that match his component functionalities and implement the service methods manipulating only elements of the exchange data model and in accordance to its definition.
Potential Impact:
Impact creation
One of the most visible trends in the IT industry in recent times has been the explosive growth of digital information. Users clearly need a way to extract relevant information quickly and easily out of this increasing amount of data — much of which is non-essential.
To contribute to that objective, VIRTUOSO has built a platform to increase productivity; to face more effectively the compliance mandates through intelligent archiving and legal technologies for discovery; to improve response agility through streamlining business processes and to enhance significantly institutional decision-making.
VIRTUOSO, targeted to end-users involved in information processing and exploitation for better informed decision-making, has developed a platform that allows organizations to plug in different tools (to discover, acquire, and organize data from open sources) and inter-operate through a standard interface, increasing security across border.
Social Impact
One of the core objectives of the European Union is to create a true area of freedom, security and justice and to provide its citizens with a high level of security.
With this goal, to EU level, currently, the exchange of information in this context is still considered to be low. Nowadays, there is availability of great amount of information of different nature in multiples repositories but, too, exists a lack of tools for the collection, processing, combining, interpreting, exchanging of these data.
For this aim, the objective of VIRTUOSO is to develop a platform that enables the collecting, processing and storage of security and open source information. The open source introduces new challenges and risks, starting from trying to define what can be categorized as open source information and its use. A critical point to take into consideration is the absence of an EU standardized and harmonized framework for privacy to rule the use (and misuse) of the information that VIRTUOSO aims at handling.
To fill this gap, the project analyzed the legal and ethical constraints that need to be taken into account in the technical design of the VIRTUOSO framework, and subsequent use thereof, thereby ensuring the social and ethical-legal embedding of the architecture and software developed by the project VIRTUOSO.
Two main outcomes produced by this work are used to create social impact with regards to privacy considerations related to VIRTUOSO:
■ Privacy Impact Assessment (PIA) (D3.1) Based on the functionalities of the VIRTUOSO architecture and the implemented demonstrators, the PIA creates the basis for the ethical and legal embedding of the architecture and tool developed in this project.
■ Code as code — demonstration, platform evaluation from a legal and ethical perspective (D3.4). This document is a legal and ethical evaluation of the demonstrations prepared in the project, in order to make the design ethical lawful (ethical and legal embedding) as well as acceptable within society (social embedding). Recommendations on how the legal and ethical requirements can be codified using privacy enhancing technologies (PET) and similar techniques are provided. Also, the document investigates the phenomena that technology can be used as an intentionally regulatory instrument by embedding norms in the technology.
The two above-listed documents have been presented to the developers within the VIRTUOSO consortium but also to an external and wider audience in a series of workshops and meetings, to ensure their awareness and to incorporate their feedback in the project tasks.
On the other hand, using Open Source technologies and defining European standards in this environment will allow developers to boost the work began by the VIRTUOSO project, as well as to evolve it so the citizen can adopt and trust in these tools, policies and actions derived from its use.
Finally, indicate that the social impact will be a collateral outcome of the implantation and use of the VIRTUOSO Platform and/or its tools in those public and security organizations, and its benefits will directly impact on the citizen that will make use of a trustworthy and safe environment.
Technological and Scientific Impact
The aim of the VIRTUOSO platform was to provide one single point of access to any tool available no matter the technology used. That is, it should be possible to pick the tools that you want, and the platform will ensure their interoperability. The platform is also able to allow end-users to build their own solution responding to the specific needs of each organisation.
Four levels of interoperability have been satisfied:
■ Infrastructure level: to ensure the reuse and the integrations of heterogeneous technologies.
■ Resource level: developing a resource respecting standard like MLIF (Multi Information Framework) ISO WD 24616
■ Knowledge level: to ensure the importation and exportation of ontologies and knowledge base the development of VKBS (processing the data gathered from open source information) based on knowledge representation standards, such as RDF, OWL, SKOS or RIF.
■ Data level: the gathered data will be converted into a standard format, for instance XML format
Don’t forget that there is a big complexity of data (many different types of data sources and formats) but the VIRTUOSO architecture supports the transformation of unstructured data into structured data. Recall the objective of this project is to promote this standardization of VIRTUOSO as the common platform.
Moreover, the technological liaison with OW2 has supposed that the main platform components (WebLab, ING Semantic Infrastructure) are on their way to be an Open Source Project under OW2.
Business impact
The European Commission says in its policies “EU Home Affairs policies can be grouped under two fields: migration and asylum, and internal security. The first includes issues related to legal and irregular migration, integration, readmission and return. The second encompasses the fight against organized crime and terrorism, police cooperation and the management of the EU's external border. The EU also actively cooperates with non-EU countries to work together on migration- and security-related challenges”
In this context, the intention behind the Virtuoso project is to learn how intelligence's professionals work and what they would expect of a software solution designed to make their daily work easier.
The VIRTUOSO platform enables intelligence organizations and commercial entities to make use of open source information to obtain intelligence that enhances their operations, be it for cost or for improved results. That is, VIRTUOSO has been developed with multiple stakeholders, such as policy makers, groups who define research programmers, experts and representatives from governmental agencies, legal advisers, technical experts but also scientists, professional active in the field, etc. the focus was that the results of the project should facilitate these groups in effective and efficient development, planning and implementation of research, actions and policies.
On the other hand, the reality is that the project activities are addressed to develop a middleware platform for the integration of tools that enables "plug-and-play" functionalities, allowing market-oriented technological solutions that will improve the ability of open source information professionals to utilize data from across the source and format spectrum in support of the decision-making process.
The main technical output expected out of the project is this technical platform, based on an open-source software solution, for the integration of tools for collection, processing, analysis, visualization and decision-support of open source information, to allow end-users to create their own customized and modular open source information management solution.
Moreover, the contact with relevant players (end-user) in these activities has been fundamental to receive feedback and, in an obvious manner, their input has been too crucial for forthcomings improvements in the VIRTUOSO framework and tools as well as for future collaboration (thinking in business terms –business impact).
In this context, the VIRTUOSO exploitation has to do with "use", as it is referred in the Grant Agreement (Annex II- General Conditions), which “means the direct or indirect utilisation of the project’s foreground in further research activities other than those covered by the project, or for developing, creating and marketing a product or process, or for creating and providing a service”. Exploitation concerns the protection of the investing parties’ interests. The objective is to make sure that the results and knowledge gained individually and collectively by the parties does not remain unused.
The exploitation model for VIRTUOSO is a question handled within task T10.4 and detailed information on VIRTUOSO exploitation model is collected in the deliverable document
Nevertheless VIRTUOSO is not conceived to be a closed product, and thus, the possibility to enhance the system envisages different business offerings after its deployment.
For example, according to the customer’s specific needs and the adoption capabilities, the consultancy (business/ technical consultancy) services could be offered as items or as package.
However, the business impact depends more on the partner nature and its business model.
For a more detailed description on the use that each of the partners of the consortium will do of VIRTUOSO results, refer to D10.5. Exploitation model.
Dissemination activities
To promote exploitation opportunities, VIRTUOSO partners have been involved in different dissemination activities, aiming relevant industry companies and public authorities.
At the same time the different partners’ exploitation models had their impact on dissemination towards the VIRTUOSO stakeholders.
Dissemination purpose is to ensure that relevant audiences understand what was done, and how they might use the lessons learned to meet their own needs. The exploitability of the VIRTUOSO results went hand in hand with an adequate dissemination strategy.
As result of different diffusion and dissemination activities carried out during the VIRTUOSO project, a number of relationships was established in order to increase the interest from the stakeholders and support as well the exploitability of the VIRTUOSO results. The same mode, some dissemination actions have resulted from establishing new links and relationships.
Maintaining a network of relevant contact points has proven useful to support dissemination activities (and vice versa) and gain visibility for VIRTUOSO results and thus reach potential end-users, that is, consumers of the VIRTUOSO-based solutions and technology providers that contribute to the VIRTUOSO approach
To develop dissemination plans at all levels and to ensure that the content of all dissemination materials met appropriate criteria – defined within VIRTUOSO project - a Dissemination Board (DB) was established
The DB was formed inside the consortium to provide an independent view of the work from the dissemination point of view, to ensure that classified or sensitive data were not released, consulting the End-User Advisory Board on this subject when necessary and to let partenrs check that their IPR and different academic or commercial interests were not affected or compromised.
A summary of the main dissemination activities is presented hereafter :
VIRTUOSO website (http://www.virtuoso.eu)
The site was and still is a dissemination tool providing public access to general information on the project (objectives, partners, workshop registration and proceedings, other upcoming events, etc.), and to its public deliverables and presentations. Also it has a restricted part for exchanging internal project information between partners.
VIRTUOSO Workshops
During the different phases of the project - over the period 2010-2013 - the consortium organised a series of workshops with different audience and scope depending on the stage and available results of the project.
There was a set of ten workshops devoted to gather inputs for the development of the VIRTUOSO platform. This was successfully achieved by inviting intelligence professionals working for ministries, law enforcement agencies, customs agencies, intelligence services, and the armed forces from all across the European Union. The intent behind the workshops was to learn how intelligence professionnals work and what they would expect of a software solution designed to make their daily work easier.
For the last stages of the project another set of ten workshops was held with the end-users to evaluate the work done and comprising the demonstration of the version V1, and V2 –final version- of VIRTUOSO.
The dissemination level ranged from awareness, in the first stages of the project, till understanding and action given the strong commitment of the end-users with the project.
The acceptance of the various workshops was extraordinary. The average number of end-users organisations per workshop was 12, with an average of more than one representative per organization. In total over 72 individuals representing 40 organizations attended these workshops. Many of those individuals participated in two or more workshops and can therefore be considered as taking active part in the development process of VIRTUOSO.
Besides those, other three dissemination specific workshops took place along the project. A first one at the end of the second year of the project, once the version V1 of the VIRTUOSO platform was available, to promote VIRTUOSO and its intermediate results.
During this dissemination workshop some VIRTUOSO end-users were presenting their involvement and their interest in VIRTUOSO (CNCA - Spanish National Centre for Antiterrorist Coordination of Spanish Ministry of Interior; SatCen – European Satellite Centre, as representative of an European institution involved from the beginning in the project; Open Source Centre, from Romania, that has shown a big interest in the project and whose representative has participated in all end-users workshops; and the Ministry of Foreign Affairs of Lithuania, as representative of a small eastern country).
Also other FP7 co-funded related projects participated in this first dissemination workshop (CAPER, RECOBIA, ePOOLICE and TELEIOS) together with Madri+d Regional Foundation as networking and socializing facilitators.
IBEROSINT, institutional intelligence community brought up around VIRTUOSO, gathering the Spanish end-users in the field of intelligence and open sources, was presented by a representative of Guardia Civil, one of its members.
A dissemination workshop on “Ethical and legal issues in the analysis of open sources”, was held in on April, 2013 where the legal and ethical issues related to the analysis and management of open source information were discussed and experiences among different FP7 European Projects (CAPER. SAPIENT) exchanged.
A large final dissemination workshop took place at the end of the project as a conference of the VIRTUOSO project whose objective was to offer the opportunity to external contributors to present services or products which could be integrated in a VIRTUOSO-based information processing system. Companies that presented their solutions were Oxford Analytica, Systran, Senseetive, Visibrain and Agnitio.
Further, the two intelligence professionals networks which supported the VIRTUOSO project, the EUROSINT Forum and the IBEROSINT Forum, were both presented. Finally, the consortium partners were also invited to showcase a specific service or tool that was developed by them in the framework of the VIRTUOSO project.
This final conference concluded with a session where the future use of the outcomes of the VIRTUOSO project by the end-user organizations was discussed.
VIRTUOSO demonstrations
Besides the main version’s demonstrations (mentioned V0, V1 and V2 ) a VIRTUOSO roadshow , set of “at home” demonstrations, was carried out at restricted - national or organisational - audiences.
This targeted action oriented dissemination implied visits to different countries to a selected audience of end-users interested in open source intelligence in the security field. Demonstrations were given to British intelligence community in London, French intelligence community in Paris, the German intelligence community in Berlin, Spanish intelligence community in Madrid, Romanian intelligence community in Bucharest, Swedish intelligence community in Stockholm, Lithuanian intelligence community in Vilnius, Polish military and civilian intelligence agencies, in Warsaw and to the NATO OSINT Group in Brussels.
Involving the mentioned Spanish IBEROSINT forum four demonstrations took place in Madrid.
Other demonstrations were given taking advantage of different international events — during the Secure Day 2011 event; during the European Information Network for International Relations and Area Studies (EINIRAS) Annual Conference — 2012; and at OW2Con’12, event Sponsored by VIRTUOSO
Other events
Information on the VIRTUOSO project has been disseminated with different reach and scope on 26 international events. This represents about 9 events per year of the projectScientific Publications
Scientific publications
A total of fifteen publications have been worked out in the VIRTUOSO project, mainly to be presented at different international conferences but also for Scientific / academic journals.
Linking with on-going initiatives
The aim was the development, maintenance and support of a network linking public authority users, OSINT research community, related projects and potential contributors (providers and integrators in this area). Liaison with other initiatives interested VIRTUOSO to get both the user community and the application developers to join the VIRTUOSO middleware effort.
Maintaining a network of relevant contact points could be used to support dissemination activities and gain visibility for VIRTUOSO results and thus reach decision makers.
Examples of this are the links maintained during the project with other research initiatives, FP7 projects as CAPER, RECOBIA, ePOOLICE, TELEIOS, SAPIENT, ARCHIMEDES, ePOOLICE. MODAP, OSMOSIS; with fora on OSINT subjects, as EUROSINT and IBEROSINT ; with technological platforms as WebLab and OW2; and Madri+d, regional foundation for the promotion of the technology research transfer to industry.
Others
On top of all stated previously, it should be mentioned, as important dissemination activities of VIRTUOSO, the publication of nine press releases relative to the project in different media in several European countries, and a set of briefings targeted to specific groups of stakeholders particularly interested in knowing on VIRTUOSO.
There was also a survey carried out by VIRTUOSO Work Package Ethics, legal and privacy issues. This survey was available to public in general through a link on the public area of the VIRTUOSO website.
It is worth to mention that different dissemination material was prepared in the form of flyers, posters, kakemono, pen drives. On this regard it can be highlighted the preparation of training material accessible on the web and two VIRTUOSO videos available through the VIRTUOSO website. One video illustrating the VIRTUOSO platform demonstration and the other showing the ontology maintenance in ITM based on the Trend mining service.
List of Websites:
Website and contact details
The VIRTUOSO web site has been operational since the beginning of the project and is accessible through the following URL: http://www.virtuoso.eu/.
The site is a dissemination tool providing public access to general information on the project (objectives, partners, workshops, other events, etc.), and to its public deliverables, presentations and other material as e.g. videos.
It also has a restricted part for exchanging internal project information between partners. In their private area there is a VIRTUOSO Website User Manual available.
Contact information can be found through the link Contact us in the public area of the website, with information about the VIRTUOSO project coordinator:
CEA List
Géraud Canet
E-mail: geraud canet@cea.fr
☎ +33 6 85 89 85 08
✉ CEA Saclay, Nano Innov / Point courrier 173 / Bâtiment 861
F - 91191 Gif-sur-Yvette - FRANCE
Also through the Legal notice link that let access the conditions and terms of use of the VIRTUOSO website: In the event of a complaint, or to request further information, the provider may be contacted in writing at virtuoso@virtuoso.eu.
VIRTUOSO provides a technical framework for the integration of tools for collection, processing, analysis and communication of open source information. "Plug and play" functionalities that improve the ability of border control, security and law enforcement professionals to use data from across the source / format spectrum in support of the decision making process are enabled by this middleware framework. As a proof of concept and to highlight the efficiency of this open-source code framework, a prototype has been built and demonstrated using operational scenarios.
The project, developed by a consortium of 16 European organisations, is co-funded under the FP7 programme of the EU and will comply with legal considerations, enforcing the principles of privacy
and data protection to ensure the interests of citizens within the European Union. The main functional components of the VIRTUOSO framework are:
- Information Gathering components: Collect the information coming from multiple supports (disks, intranet, the Internet...) multiple formats (paper, digital, analogical...) and of multiple natures
(textual, audio and visual (image and video)), and then convert data into standard format. For instance paper documents can be scanned and translated via OCR API into XML output format, in
the same way the speech audio documents are converted, using speech to text API, into XML format.
- Information Processing and Structuring components: Structuring Open Source Information to Support Intelligence Analysis. It deals with information reduction and structured based on event and
information extraction. It comprises information filtering and categorization, Information Retrieval, Entity and event extraction, Information summarization.
- Knowledge Management components: These services deal with the representation and the exploitation of the knowledge built from the extracted information; A VIRTUOSO Knowledge Base
(VKB) has also been developed in order to provide a uniform data model satisfying all the needs for representing and using operational and conceptual knowledge
- Decision support components: Provide a substantial aid to the decision makers by integrating various sources of information and by developing tools to exploit the knowledge stored in the VKB and components for Situation Assessment, Scenario Building and Misinformation detection.
- Visualization components: Provide an intelligent access to visualize relevant knowledge, and aid the process of structuring the work and the decisions upon the extracted information.
- Evaluation components: Evaluate atomic services and processing chains to compare the capability and performance of different services or chain configurations in specific task such as: named entity, extraction, speech to text, etc.
To deal with the integration of these different components, the proposed framework is composed of 4 layers:
- Processing framework: collect, analysis, indexing, etc.
- Decision support framework: knowledge management, reasoning, etc.
- Visualization framework: unified HMIs, interoperability mechanisms, etc.
- Evaluation framework: mechanisms for the evaluation of atomic services and processing chains
Project Context and Objectives:
Open Source Intelligence (OSINT) is an information processing discipline that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence.
The previous trend has been for intelligence agencies to primarily focus on obtaining information that is not available to ordinary citizens, though networks of agents, or the monitoring of electronic communications. OSINT is to play a pivotal role in today’s information society with the ever-growing amount of information publicly available.
Benefits
• Speed and coverage
As Stephen C. Mercato wrote in “Reexamining the distinction between open source information and secrets”, ‘There are more bloggers, journalists, pundits, television reporters, and think tankers in the words than there are case officers” .
• Quantity
OSINT is drawn from publicly available material including, the Internet, traditional mass media ‘television radio, newspapers, magazines), specialized publications (scientific and specialized journals; conferences proceedings, think thank studies), pictures and photos, geospatial information (maps, commercial imagery).
• Quality
“The OSINT target is immense. Engineers attend conferences; scientists publish scholarly articles; manufacturers build production lines; bureaucrats issue guidelines; and traders print brochures for prospective clients. Many paper trails wind around the world beyond whatever may surface in the media.”
• Ease of use & extended monitoring
• Cost & Risk management
Challenges
The key role of Internet as a vehicle for spreading information is undeniable. One of the main challenges is to manage the enormous volume and flow of information. Traditional media also use the same information and validation channels as their audience. This increases the possibility to influence because transmitters and receivers tend to become intertwined.
Therefore, the second challenge is for intelligence analysts to identify all possible actionable intelligence and analyze it. If open source information is not classified, it does not mean however that it is easy to find or even understand and interpret.
Finally, in OSINT, the ability of the intelligence analyst to analyze is at the heart of the discipline. The difference between the two classifications (Open vs Classified / Secret) depends as much on the ability to analyze as on the level of the information received. However, it mostly depends on the ability of the user to disseminate, re-arrange, and interpret information.
The main functional components of the VIRTUOSO framework are:
- Information Gathering components: Collect the information coming from multiple supports (disks, intranet, the Internet...) multiple formats (paper, digital, analogical...) and of multiple natures
(textual, audio and visual (image and video)), and then convert data into standard format. For instance paper documents can be scanned and translated via OCR API into XML output format, in
the same way the speech audio documents are converted, using speech to text API, into XML format.
- Information Processing and Structuring components: Structuring Open Source Information to Support Intelligence Analysis. It deals with information reduction and structured based on event and
information extraction. It comprises information filtering and categorization, Information Retrieval, Entity and event extraction, Information summarization.
- Knowledge Management components: These services deal with the representation and the exploitation of the knowledge built from the extracted information; A VIRTUOSO Knowledge Base
(VKB) has also been developed in order to provide a uniform data model satisfying all the needs for representing and using operational and conceptual knowledge
- Decision support components: Provide a substantial aid to the decision makers by integrating various sources of information and by developing tools to exploit the knowledge stored in the VKB and components for Situation Assessment, Scenario Building and Misinformation detection.
- Visualization components: Provide an intelligent access to visualize relevant knowledge, and aid the process of structuring the work and the decisions upon the extracted information.
- Evaluation components: Evaluate atomic services and processing chains to compare the capability and performance of different services or chain configurations in specific task such as: named entity, extraction, speech to text, etc.
To deal with the integration of these different components, the proposed framework is composed of 4 layers:
- Processing framework: collect, analysis, indexing, etc.
- Decision support framework: knowledge management, reasoning, etc.
- Visualization framework: unified HMIs, interoperability mechanisms, etc.
- Evaluation framework: mechanisms for the evaluation of atomic services and processing chains
Project Results:
Definition of the core platform
The Virtuoso core platform consists of several open source based components:
The main contribution in the platform is Weblab, the other part is ING. All projects members worked in synergy to embed all parts of the platform in one. As a result, Weblab C++, evaluation framework and presentation framework are integrated into Weblab repositeries as part of Weblab. In this document, we will refer the the platform as the association of Weblab and ING.
Weblab: an open source platform for supporting SoA applications
Virtuoso platform is composed of four coherent groups of components, named frameworks. These frameworks are detailed below.
The processing orchestration framework: WebLab is the chosen platform for the VIRTUOSO platform and used as basis for the VIRTUOSO processing framework. WebLab provides a set of tools and methods that aim at building information systems for intelligence applications in the economy, strategy and military domains.
The evaluation framework: Based on the WebLab platform, it allows evaluating WebLab processing components in terms of the quality of the underlying algorithm. For example, a named entities extraction service will be evaluated in terms of precision and recall of the extracted entities. The evaluation services will wrap evaluation campaigns based tools inside a WebLab interface.
The decision support framework: The ING Semantic Middleware, ING for short, was chosen to provide the basis for the VIRTUOSO decision support framework in the VIRTUOSO platform. It provides functionalities to interact with the KB. For the VIRTUOSO demonstrator, the KB solution will be ITM (Intelligent Topic Manager), provided by MONDECA. The ING core architecture implementation already provides a generic driver for the ITM access, but for the VIRTUOSO demonstrator, it would be required to develop a specific VIRTUOSO meta-model ITM driver.
The unified presentation framework: it ensures the standardisation of the user interfaces of the application upon the platform. This covers three different aspects: graphical aspects, integration methodology and standardization of communication between user interface fragments. As part of the integration in the global architecture the presentation framework relies on a web interface hosted on a web portal and each interface component will be integrated using common technologies.
WebLab: an open source platform for supporting OSINT applications
The WebLab framework is an open source technical baseline acting as a runtime environment for unstructured information processing services. It provides a set of tools and methods that provide easy and flexible building of media-mining applications (OSINT, business intelligence, etc). Typically, WebLab handles multimedia and unstructured data (text, image, audio and video) through heterogeneous business components providing services.
The WebLab platform relies on a SOA (Service Oriented Architectures) as the core paradigm for the design and integration of components. The high-level functions, offered to users through applications, are achieved by chaining together services and calling them in the right sequence. As a consequence, the service definition and conception is a key feature in the platform.
The WebLab technical Architecture can be seen as a multi-layer architecture.
Having a top down view, the WebLab architecture consists on the following layers:
• The access layer provides (i) graphical user interfaces (GUIs) that enable “final” users to access services in a hidden way, (ii) a monitoring application that enables "admin" users to control and check the execution of the processing chains and(iii) an administration application that includes tools supporting business workflow deployment and configuration and enabling “architect" users to define service chains.
• The process layer or orchestration layer consists in an orchestrator that acts as an intermediary between services in order to perform a business process.
• The service bus layer is in charge of communication and distribution of messages between services in accordance to the request expressed by the orchestrator. It allows abstraction of services localization and implementation as well as abstraction of communication protocol.
• The service layer contains standardized services that are developed upon the WebLab Data Exchange Model and implement WebLab Generic Interfaces. These services act as connectors to technical & business components.
• The component layer is where developed components (that are hidden behind the standardized services) are located.
• The data layer contains (a) repositories used by the platform for the storage of original content of documents and their description according to the WebLab Model (after processing), (b) indexes and (c) knowledge bases.
Each component integrated in the WebLab platform is developed upon the WebLab Data Exchange Model that will describe the resources used within a WebLab application, and implements one (or several) WebLab Generic interface(s) that defines their input and output in terms of WebLab data exchange model objects.
WebLab has a number of significant benefits:
o It allows fast creation of processing chains to meet any user needs.
o It provides a flexible solution where components can be interchangeable according to the requirements of the specific tasks.
o It enhances interoperability and intercommunication of heterogeneous systems, services and processes.
o Provides the use of standard security protocols to allow protection of confidentiality and integrity of information across the system.
o It relies on a scalable architecture allowing the integration of mass data analysis components.
o It uses standards for media-mining and data fusion (XML, OWL, OGC, RDF, SOA, etc.).
ING: a decision support middleware
ING Semantic Infrastructure is an Open Source environment (LGPL2 licence) developed by Thales Communications and Security (TCS). ING Semantic Infrastructure is the heart of the Decision Support part of VIRTUOSO, as it is the integration platform used by user in the loop VIRTUOSO applications working on the Knowledge Base. ING was initiated during the design phase of French national security project (2006), and enhanced in the OSEMINTI (EDA 2007-2009) and VIRTUOSO (EU) R&D projects. It is now contributing to Thales solutions in the Security and Intelligence domains.
The purpose of ING Semantic Infrastructure is to facilitate the integration of Decision Support (user in the loop) components from various origins, based on the semantic technology. It brings the flexibility and expressivity of semantic technology to the world of decision making.
The semantic representation relies on the representation of knowledge as triples (subject – verb – object).
ING is compatible with the RDF and TOPIC MAPS semantic representation standards issued respectively by W3C and ISO.
Recognized for its flexibility, semantic representation is increasingly used in decision making systems. ING Semantic Infrastructure proposes a generic environment to develop Decision Making application based on semantic technology.
The main components of ING Abstract Semantic Repository are:
• ING abstract semantic repository which provides APIs (query, update, model query) to address any repository as a semantic repository.
• ING Extensions which offers common services dedicated to data exchanges (the “basket”), traceability, and generic interfaces to operational functionality such as similarity and data fusion.
• ING Connectors offers a connector to connect ING to Mondeca ITM repository, or for memory storage, soon a RDF connector will be released by Thales.
ING Semantic Infrastructure & basic applications
The major benefits brought by ING Semantic Infrastructure are:
• to make application components independent from the repository actually used,
• to provide model adaptability to the components,
• to introduce a specific meta-model, which differentiates operational links (shown to the users) from technical links (used only by applications and hidden from the users) avoids to overflow end-users with useless information.
By its open architecture, ING Semantic Infrastructure makes the integration of new Decision Support applicative modules easy.
A set of decision support applications (entity editor, model editor, collective fusion) developed by Thales on top of ING Semantic Infrastructure are part of the ING open source release. In addition, the visualisation modules (Graph Viewer and Tabular viewer) developed by ATOS in VIRTUOSO are ING compatible and will soon be released as open source software.
Evaluation framework
An important part of an intelligence analyst's work revolves around using the right tools for the right task. As such, Virtuoso aims to respond to the need to have an architecture and software solution geared towards automated comparison of software offered by vendors. The project answered this need in two parts: benchmarking and evaluation.
Benchmarking
Benchmarking guidelines were established to rate the following technical elements: technical readiness, ease of integration, documentation availability, etc. The complete guidelines are part of Output 2.7 (Task 2.4) – “Benchmarking Guidelines”. These are aimed at technology providers, for them to be able to provide standardized readiness information to intelligence agencies.
Evaluation
Evaluation seeks to rate the quality of data processing components output. The method chosen by the Virtuoso project is to encourage the use of international evaluation campaigns in order to rate specific tasks. To this end, the consortium has developed specifications for the integration of any evaluation campaign into the Evaluation Framework using WebLab specifications.
Evaluation campaign
An evaluation campaign is made of a kit designed to judge the performance of a given type of tool (named entity extraction, translation, summarization, categorization, etc) over a specific set of data (news reports, scientific papers, tweets, etc).
Well-known evaluation campaigns include CONLL and MUC, for text-based information processing such as named entities extraction or translation, or TREC for text and multimedia information retrieval .
The following is typically contained in an evaluation campaign kit:
• Evaluation data set (or Test data). A set of documents to be run through the component to be evaluated.
• Ground truth (or Gold standard). The best possible expected outcome for the evaluation data set. Usually this is obtained through manual or semi-automatic annotation.
• Evaluation tool (or Scoring tool). A tool which compares the ground truth with the output of the component and computes a set of metrics. These metrics, defined for the purposes of the campaign, are the quantitative result of the evaluation. Each tool has its own requirements in terms of I/O.
• Training data. Some evaluation campaigns require the use of specific training data on the component before processing the evaluation data set. In these cases training data is included. However components which do not require training do not need to use this data.
An evaluation is run over a component through the following steps:
• (optional) The component to be evaluated is trained using the training data provided by the evaluation campaign.
• The component processes the evaluation data set. The outputs are converted into the input format expected by the evaluation tool.
• The evaluation tool processes the component output and compares it to the ground truth. Specific metrics are produced to quantify the quality of processing over the data.
Evaluation framework
The evaluation framework seeks to give the ability to integrate any evaluation campaign into WebLab in order to evaluate the various components used in OSINT tasks. The consortium demonstrated the feasibility of the evaluation framework using data from the CONLL 2003 evaluation campaign to evaluate named entity extraction components used in Virtuoso.
The specifications of the Evaluation Framework are part of Output 2.2 – “Overall Architecture design v1” and have been released as open source as online tutorials in order to facilitate integration.
A practice the Virtuoso project sought to encourage is for intelligence agencies to define their own evaluation campaigns (data sets, metrics). While recognized as a standard, international evaluation campaigns often have the disadvantages of
• being too generic,
• focusing on a specific type of media such as newspapers, tweets or scientific papers,
• answering the needs of a specific research agency (ex. DARPA).
Thus it may be more beneficial for intelligence agencies to build their own evaluation campaigns with the help of scientific experts.
Integration process
The proposed architecture is service oriented (SOA). Its aim is to avoid the dependency between the components and dependency to the platform. Each software component in VIRTUOSO is developed as an independent service both on the processing or on the user interface level. The basic idea is that each component consume and/or produce data using the common exchange model but that one-to-one communication between components should be avoided except data repository. Thus each component is independent from the real implementation and location of the component producing the data for them or consuming the data they produce. The same process is used for the user interface level.
Thanks to this architecture, the developer partners have done minor adaptation of their own component to be “VIRTUOSO ready”.
The major steps of this integration are:
1. Encapsulation and validation of technical components developed/provided in the wp-4-5-6-7 into web services w.r.t WP2 specifications. This work is done in each technical workpackge and relied on the integration factories developed in the WP8.
2. Deployment of services on internal servers or on CASSIDIAN's servers.
Test and robustification of each web services (to address concurrency requests and scalability for example).
3. Integration into the demonstrator of each web services (processing chain, interoperability with other services, etc.). This work is done in the WP8.
4. Test and validation of the integrated service. This work is done in WP8.
The WP2 has provided the technical specification which contain: a set of a generic interfaces and their IOPE(to be implemented by services), an exchange data model, an annotation model, etc. To do the step 1, the developer should select generic service interfaces that match his component functionalities and implement the service methods manipulating only elements of the exchange data model and in accordance to its definition.
Potential Impact:
Impact creation
One of the most visible trends in the IT industry in recent times has been the explosive growth of digital information. Users clearly need a way to extract relevant information quickly and easily out of this increasing amount of data — much of which is non-essential.
To contribute to that objective, VIRTUOSO has built a platform to increase productivity; to face more effectively the compliance mandates through intelligent archiving and legal technologies for discovery; to improve response agility through streamlining business processes and to enhance significantly institutional decision-making.
VIRTUOSO, targeted to end-users involved in information processing and exploitation for better informed decision-making, has developed a platform that allows organizations to plug in different tools (to discover, acquire, and organize data from open sources) and inter-operate through a standard interface, increasing security across border.
Social Impact
One of the core objectives of the European Union is to create a true area of freedom, security and justice and to provide its citizens with a high level of security.
With this goal, to EU level, currently, the exchange of information in this context is still considered to be low. Nowadays, there is availability of great amount of information of different nature in multiples repositories but, too, exists a lack of tools for the collection, processing, combining, interpreting, exchanging of these data.
For this aim, the objective of VIRTUOSO is to develop a platform that enables the collecting, processing and storage of security and open source information. The open source introduces new challenges and risks, starting from trying to define what can be categorized as open source information and its use. A critical point to take into consideration is the absence of an EU standardized and harmonized framework for privacy to rule the use (and misuse) of the information that VIRTUOSO aims at handling.
To fill this gap, the project analyzed the legal and ethical constraints that need to be taken into account in the technical design of the VIRTUOSO framework, and subsequent use thereof, thereby ensuring the social and ethical-legal embedding of the architecture and software developed by the project VIRTUOSO.
Two main outcomes produced by this work are used to create social impact with regards to privacy considerations related to VIRTUOSO:
■ Privacy Impact Assessment (PIA) (D3.1) Based on the functionalities of the VIRTUOSO architecture and the implemented demonstrators, the PIA creates the basis for the ethical and legal embedding of the architecture and tool developed in this project.
■ Code as code — demonstration, platform evaluation from a legal and ethical perspective (D3.4). This document is a legal and ethical evaluation of the demonstrations prepared in the project, in order to make the design ethical lawful (ethical and legal embedding) as well as acceptable within society (social embedding). Recommendations on how the legal and ethical requirements can be codified using privacy enhancing technologies (PET) and similar techniques are provided. Also, the document investigates the phenomena that technology can be used as an intentionally regulatory instrument by embedding norms in the technology.
The two above-listed documents have been presented to the developers within the VIRTUOSO consortium but also to an external and wider audience in a series of workshops and meetings, to ensure their awareness and to incorporate their feedback in the project tasks.
On the other hand, using Open Source technologies and defining European standards in this environment will allow developers to boost the work began by the VIRTUOSO project, as well as to evolve it so the citizen can adopt and trust in these tools, policies and actions derived from its use.
Finally, indicate that the social impact will be a collateral outcome of the implantation and use of the VIRTUOSO Platform and/or its tools in those public and security organizations, and its benefits will directly impact on the citizen that will make use of a trustworthy and safe environment.
Technological and Scientific Impact
The aim of the VIRTUOSO platform was to provide one single point of access to any tool available no matter the technology used. That is, it should be possible to pick the tools that you want, and the platform will ensure their interoperability. The platform is also able to allow end-users to build their own solution responding to the specific needs of each organisation.
Four levels of interoperability have been satisfied:
■ Infrastructure level: to ensure the reuse and the integrations of heterogeneous technologies.
■ Resource level: developing a resource respecting standard like MLIF (Multi Information Framework) ISO WD 24616
■ Knowledge level: to ensure the importation and exportation of ontologies and knowledge base the development of VKBS (processing the data gathered from open source information) based on knowledge representation standards, such as RDF, OWL, SKOS or RIF.
■ Data level: the gathered data will be converted into a standard format, for instance XML format
Don’t forget that there is a big complexity of data (many different types of data sources and formats) but the VIRTUOSO architecture supports the transformation of unstructured data into structured data. Recall the objective of this project is to promote this standardization of VIRTUOSO as the common platform.
Moreover, the technological liaison with OW2 has supposed that the main platform components (WebLab, ING Semantic Infrastructure) are on their way to be an Open Source Project under OW2.
Business impact
The European Commission says in its policies “EU Home Affairs policies can be grouped under two fields: migration and asylum, and internal security. The first includes issues related to legal and irregular migration, integration, readmission and return. The second encompasses the fight against organized crime and terrorism, police cooperation and the management of the EU's external border. The EU also actively cooperates with non-EU countries to work together on migration- and security-related challenges”
In this context, the intention behind the Virtuoso project is to learn how intelligence's professionals work and what they would expect of a software solution designed to make their daily work easier.
The VIRTUOSO platform enables intelligence organizations and commercial entities to make use of open source information to obtain intelligence that enhances their operations, be it for cost or for improved results. That is, VIRTUOSO has been developed with multiple stakeholders, such as policy makers, groups who define research programmers, experts and representatives from governmental agencies, legal advisers, technical experts but also scientists, professional active in the field, etc. the focus was that the results of the project should facilitate these groups in effective and efficient development, planning and implementation of research, actions and policies.
On the other hand, the reality is that the project activities are addressed to develop a middleware platform for the integration of tools that enables "plug-and-play" functionalities, allowing market-oriented technological solutions that will improve the ability of open source information professionals to utilize data from across the source and format spectrum in support of the decision-making process.
The main technical output expected out of the project is this technical platform, based on an open-source software solution, for the integration of tools for collection, processing, analysis, visualization and decision-support of open source information, to allow end-users to create their own customized and modular open source information management solution.
Moreover, the contact with relevant players (end-user) in these activities has been fundamental to receive feedback and, in an obvious manner, their input has been too crucial for forthcomings improvements in the VIRTUOSO framework and tools as well as for future collaboration (thinking in business terms –business impact).
In this context, the VIRTUOSO exploitation has to do with "use", as it is referred in the Grant Agreement (Annex II- General Conditions), which “means the direct or indirect utilisation of the project’s foreground in further research activities other than those covered by the project, or for developing, creating and marketing a product or process, or for creating and providing a service”. Exploitation concerns the protection of the investing parties’ interests. The objective is to make sure that the results and knowledge gained individually and collectively by the parties does not remain unused.
The exploitation model for VIRTUOSO is a question handled within task T10.4 and detailed information on VIRTUOSO exploitation model is collected in the deliverable document
Nevertheless VIRTUOSO is not conceived to be a closed product, and thus, the possibility to enhance the system envisages different business offerings after its deployment.
For example, according to the customer’s specific needs and the adoption capabilities, the consultancy (business/ technical consultancy) services could be offered as items or as package.
However, the business impact depends more on the partner nature and its business model.
For a more detailed description on the use that each of the partners of the consortium will do of VIRTUOSO results, refer to D10.5. Exploitation model.
Dissemination activities
To promote exploitation opportunities, VIRTUOSO partners have been involved in different dissemination activities, aiming relevant industry companies and public authorities.
At the same time the different partners’ exploitation models had their impact on dissemination towards the VIRTUOSO stakeholders.
Dissemination purpose is to ensure that relevant audiences understand what was done, and how they might use the lessons learned to meet their own needs. The exploitability of the VIRTUOSO results went hand in hand with an adequate dissemination strategy.
As result of different diffusion and dissemination activities carried out during the VIRTUOSO project, a number of relationships was established in order to increase the interest from the stakeholders and support as well the exploitability of the VIRTUOSO results. The same mode, some dissemination actions have resulted from establishing new links and relationships.
Maintaining a network of relevant contact points has proven useful to support dissemination activities (and vice versa) and gain visibility for VIRTUOSO results and thus reach potential end-users, that is, consumers of the VIRTUOSO-based solutions and technology providers that contribute to the VIRTUOSO approach
To develop dissemination plans at all levels and to ensure that the content of all dissemination materials met appropriate criteria – defined within VIRTUOSO project - a Dissemination Board (DB) was established
The DB was formed inside the consortium to provide an independent view of the work from the dissemination point of view, to ensure that classified or sensitive data were not released, consulting the End-User Advisory Board on this subject when necessary and to let partenrs check that their IPR and different academic or commercial interests were not affected or compromised.
A summary of the main dissemination activities is presented hereafter :
VIRTUOSO website (http://www.virtuoso.eu)
The site was and still is a dissemination tool providing public access to general information on the project (objectives, partners, workshop registration and proceedings, other upcoming events, etc.), and to its public deliverables and presentations. Also it has a restricted part for exchanging internal project information between partners.
VIRTUOSO Workshops
During the different phases of the project - over the period 2010-2013 - the consortium organised a series of workshops with different audience and scope depending on the stage and available results of the project.
There was a set of ten workshops devoted to gather inputs for the development of the VIRTUOSO platform. This was successfully achieved by inviting intelligence professionals working for ministries, law enforcement agencies, customs agencies, intelligence services, and the armed forces from all across the European Union. The intent behind the workshops was to learn how intelligence professionnals work and what they would expect of a software solution designed to make their daily work easier.
For the last stages of the project another set of ten workshops was held with the end-users to evaluate the work done and comprising the demonstration of the version V1, and V2 –final version- of VIRTUOSO.
The dissemination level ranged from awareness, in the first stages of the project, till understanding and action given the strong commitment of the end-users with the project.
The acceptance of the various workshops was extraordinary. The average number of end-users organisations per workshop was 12, with an average of more than one representative per organization. In total over 72 individuals representing 40 organizations attended these workshops. Many of those individuals participated in two or more workshops and can therefore be considered as taking active part in the development process of VIRTUOSO.
Besides those, other three dissemination specific workshops took place along the project. A first one at the end of the second year of the project, once the version V1 of the VIRTUOSO platform was available, to promote VIRTUOSO and its intermediate results.
During this dissemination workshop some VIRTUOSO end-users were presenting their involvement and their interest in VIRTUOSO (CNCA - Spanish National Centre for Antiterrorist Coordination of Spanish Ministry of Interior; SatCen – European Satellite Centre, as representative of an European institution involved from the beginning in the project; Open Source Centre, from Romania, that has shown a big interest in the project and whose representative has participated in all end-users workshops; and the Ministry of Foreign Affairs of Lithuania, as representative of a small eastern country).
Also other FP7 co-funded related projects participated in this first dissemination workshop (CAPER, RECOBIA, ePOOLICE and TELEIOS) together with Madri+d Regional Foundation as networking and socializing facilitators.
IBEROSINT, institutional intelligence community brought up around VIRTUOSO, gathering the Spanish end-users in the field of intelligence and open sources, was presented by a representative of Guardia Civil, one of its members.
A dissemination workshop on “Ethical and legal issues in the analysis of open sources”, was held in on April, 2013 where the legal and ethical issues related to the analysis and management of open source information were discussed and experiences among different FP7 European Projects (CAPER. SAPIENT) exchanged.
A large final dissemination workshop took place at the end of the project as a conference of the VIRTUOSO project whose objective was to offer the opportunity to external contributors to present services or products which could be integrated in a VIRTUOSO-based information processing system. Companies that presented their solutions were Oxford Analytica, Systran, Senseetive, Visibrain and Agnitio.
Further, the two intelligence professionals networks which supported the VIRTUOSO project, the EUROSINT Forum and the IBEROSINT Forum, were both presented. Finally, the consortium partners were also invited to showcase a specific service or tool that was developed by them in the framework of the VIRTUOSO project.
This final conference concluded with a session where the future use of the outcomes of the VIRTUOSO project by the end-user organizations was discussed.
VIRTUOSO demonstrations
Besides the main version’s demonstrations (mentioned V0, V1 and V2 ) a VIRTUOSO roadshow , set of “at home” demonstrations, was carried out at restricted - national or organisational - audiences.
This targeted action oriented dissemination implied visits to different countries to a selected audience of end-users interested in open source intelligence in the security field. Demonstrations were given to British intelligence community in London, French intelligence community in Paris, the German intelligence community in Berlin, Spanish intelligence community in Madrid, Romanian intelligence community in Bucharest, Swedish intelligence community in Stockholm, Lithuanian intelligence community in Vilnius, Polish military and civilian intelligence agencies, in Warsaw and to the NATO OSINT Group in Brussels.
Involving the mentioned Spanish IBEROSINT forum four demonstrations took place in Madrid.
Other demonstrations were given taking advantage of different international events — during the Secure Day 2011 event; during the European Information Network for International Relations and Area Studies (EINIRAS) Annual Conference — 2012; and at OW2Con’12, event Sponsored by VIRTUOSO
Other events
Information on the VIRTUOSO project has been disseminated with different reach and scope on 26 international events. This represents about 9 events per year of the projectScientific Publications
Scientific publications
A total of fifteen publications have been worked out in the VIRTUOSO project, mainly to be presented at different international conferences but also for Scientific / academic journals.
Linking with on-going initiatives
The aim was the development, maintenance and support of a network linking public authority users, OSINT research community, related projects and potential contributors (providers and integrators in this area). Liaison with other initiatives interested VIRTUOSO to get both the user community and the application developers to join the VIRTUOSO middleware effort.
Maintaining a network of relevant contact points could be used to support dissemination activities and gain visibility for VIRTUOSO results and thus reach decision makers.
Examples of this are the links maintained during the project with other research initiatives, FP7 projects as CAPER, RECOBIA, ePOOLICE, TELEIOS, SAPIENT, ARCHIMEDES, ePOOLICE. MODAP, OSMOSIS; with fora on OSINT subjects, as EUROSINT and IBEROSINT ; with technological platforms as WebLab and OW2; and Madri+d, regional foundation for the promotion of the technology research transfer to industry.
Others
On top of all stated previously, it should be mentioned, as important dissemination activities of VIRTUOSO, the publication of nine press releases relative to the project in different media in several European countries, and a set of briefings targeted to specific groups of stakeholders particularly interested in knowing on VIRTUOSO.
There was also a survey carried out by VIRTUOSO Work Package Ethics, legal and privacy issues. This survey was available to public in general through a link on the public area of the VIRTUOSO website.
It is worth to mention that different dissemination material was prepared in the form of flyers, posters, kakemono, pen drives. On this regard it can be highlighted the preparation of training material accessible on the web and two VIRTUOSO videos available through the VIRTUOSO website. One video illustrating the VIRTUOSO platform demonstration and the other showing the ontology maintenance in ITM based on the Trend mining service.
List of Websites:
Website and contact details
The VIRTUOSO web site has been operational since the beginning of the project and is accessible through the following URL: http://www.virtuoso.eu/.
The site is a dissemination tool providing public access to general information on the project (objectives, partners, workshops, other events, etc.), and to its public deliverables, presentations and other material as e.g. videos.
It also has a restricted part for exchanging internal project information between partners. In their private area there is a VIRTUOSO Website User Manual available.
Contact information can be found through the link Contact us in the public area of the website, with information about the VIRTUOSO project coordinator:
CEA List
Géraud Canet
E-mail: geraud canet@cea.fr
☎ +33 6 85 89 85 08
✉ CEA Saclay, Nano Innov / Point courrier 173 / Bâtiment 861
F - 91191 Gif-sur-Yvette - FRANCE
Also through the Legal notice link that let access the conditions and terms of use of the VIRTUOSO website: In the event of a complaint, or to request further information, the provider may be contacted in writing at virtuoso@virtuoso.eu.