Periodic Reporting for period 2 - IntelComp (A Competitive Intelligence Cloud/HPC Platform for AI-based STI Policy Making)
Reporting period: 2022-01-01 to 2023-12-31
IntelComp is an AI-powered Cloud Platform for Public Administration in the field of Science, Technology and Innovation (STI) policy. Policy stakeholders are also targeted including Industry, Academia and citizens to ensure the principle of policy “co-creation”. IntelComp provides AI-tools to assist in "data-driven" and "evidence based" policy making along the policy cycle: agenda setting, implementation, monitoring and evaluation. Based on knowledge extraction from large data sources, the IntelComp Platform is a High-Performance Computing platform and compatible with the European Open Science Infrastructure (OpenAire) and the European Open Science Cloud (EOSC).
"Fostering R&I across the EU" is the most important policy challenge for 97% of respondents to the cluster-based public consultation on EU funds in the area of investment, research & innovation, SMEs and single market”.
"About two-thirds of Europe's economic growth over the last decades has been driven by R&I. R&I supports the creation of new and better jobs and the development of knowledge-intensive activities, which account for more than 33% of total employment in Europe (European Commission Working Document, 2018).
Public administrations at all levels, the STI community, local actors, civil society and citizens (EC, National, Regional, Local) produce a great amount of dynamic, multilingual and heterogeneous data. Analyzing this data is crucial for evidence-based policy making. Hence, it demands a series of actions that to-date have been mostly performed using classical statistical approaches. This approach doesn't usually respond to the rapid pace in research and innovation. A need for new dynamic approaches able to analyze data from different sources and discover the underlying semantics would give insights to ground the formulation, implementation and monitoring of STI policies on evidence.
With co-creation and "human-in-the-loop" in the core, IntelComp adopted the Public-Private-People Partnerships living labs approach, engaging public policy makers, industry, academia, SMEs, local actors, civil society and citizens to co-create STI policies as well as to co-create IntelComp tools. IntelComp run 3 pilots targeting domains aligned with the EU Agenda and the HE Missions: Artificial Intelligence, Health (Cancer) and Climate Change.
The project has contributed a series of software applications and methodologies for policy-making using Natural Language Processing/AI:
- The IntelComp Integrated Solution encompasses 6 tools for STI policymaking: STI Dataspace, Interactive Model Trainer, STI Viewer, STI Policy Participation Portal, Evaluation Workbench, and Graph Explorer.
- A very detailed methodology to formulate measurements for STI policy questions at all stages of the policy cycle.
- User-in-the-loop curation and fine-tuning of Machine Learning models to gain the confidence of the final users.
- The co-creation approach for development of user-friendly tools.
"Fostering R&I across the EU" is the most important policy challenge for 97% of respondents to the cluster-based public consultation on EU funds in the area of investment, research & innovation, SMEs and single market”.
"About two-thirds of Europe's economic growth over the last decades has been driven by R&I. R&I supports the creation of new and better jobs and the development of knowledge-intensive activities, which account for more than 33% of total employment in Europe (European Commission Working Document, 2018).
Public administrations at all levels, the STI community, local actors, civil society and citizens (EC, National, Regional, Local) produce a great amount of dynamic, multilingual and heterogeneous data. Analyzing this data is crucial for evidence-based policy making. Hence, it demands a series of actions that to-date have been mostly performed using classical statistical approaches. This approach doesn't usually respond to the rapid pace in research and innovation. A need for new dynamic approaches able to analyze data from different sources and discover the underlying semantics would give insights to ground the formulation, implementation and monitoring of STI policies on evidence.
With co-creation and "human-in-the-loop" in the core, IntelComp adopted the Public-Private-People Partnerships living labs approach, engaging public policy makers, industry, academia, SMEs, local actors, civil society and citizens to co-create STI policies as well as to co-create IntelComp tools. IntelComp run 3 pilots targeting domains aligned with the EU Agenda and the HE Missions: Artificial Intelligence, Health (Cancer) and Climate Change.
The project has contributed a series of software applications and methodologies for policy-making using Natural Language Processing/AI:
- The IntelComp Integrated Solution encompasses 6 tools for STI policymaking: STI Dataspace, Interactive Model Trainer, STI Viewer, STI Policy Participation Portal, Evaluation Workbench, and Graph Explorer.
- A very detailed methodology to formulate measurements for STI policy questions at all stages of the policy cycle.
- User-in-the-loop curation and fine-tuning of Machine Learning models to gain the confidence of the final users.
- The co-creation approach for development of user-friendly tools.
IntelComp met its objectives at the conclusion of the project. Key Performance Indicators (KPIs) set for the whole project lifetime, were mostly achieved, outperforming many of the them, and falling short in just a few of them due to technical implementation decisions that proved beneficial and did not affect the project outcome.
IntelComp main results can be summarised as follows:
1) IntelComp defined a “data-driven” and “knowledge-based” framework for STI policy modelling. It identified a set of policy questions throughout the policy cycle and across different domains. It also identified useful measures and indicators. This was achieved through engaging Public Administrations and policy stakeholders in consultation workshops. Partners representing the STI policy community and the technical partners did an exhaustive interdisciplinary work to define to what extent AI tools could assist in responding the policy questions and which data is needed.
2) An STI Data Lake with regular updates of data facilitating the implementation of the rest of the analysis workflows
3) Implementation of machine learning services, many of which were published in the open:
- Natural Language Processing Pipelines
- Domain specific Machine Translation systems
- Domain classification service
- General purpose automatic classification toolbox based on transformers
- Field of Science (FoS) and Sustainable Development Goals (SDG) classification services
- Topic Modeling service
- Time Analysis service
- Graph Calculation and Analysis service
4) Development and deployment of IntelComp tools:
- Interactive Model Trainer
- STI Viewer
- Participation Portal
- Evaluation Workbench
- Auxiliary Tools: Data catalogue, Graph Explorer
5) Living Labs Implementation
- Refinement of the theoretical framework for the requirements and needs of each LL
- Operationalisation of IntelComp tools through the implementation of relevant use cases
- Diffusion of the project results and achievements to Public Administrations and STI policy stakeholders actively engaged in the use cases of the three Living Labs
A long list of different business scenarios has been gathered and iterated with partners for the future exploitation of the tools and the IntelComp platform as a whole. Project results have also been disseminated through scientific publications, policy briefs, project reports and a large number of LLs events targeted to relevant stakeholders.
IntelComp main results can be summarised as follows:
1) IntelComp defined a “data-driven” and “knowledge-based” framework for STI policy modelling. It identified a set of policy questions throughout the policy cycle and across different domains. It also identified useful measures and indicators. This was achieved through engaging Public Administrations and policy stakeholders in consultation workshops. Partners representing the STI policy community and the technical partners did an exhaustive interdisciplinary work to define to what extent AI tools could assist in responding the policy questions and which data is needed.
2) An STI Data Lake with regular updates of data facilitating the implementation of the rest of the analysis workflows
3) Implementation of machine learning services, many of which were published in the open:
- Natural Language Processing Pipelines
- Domain specific Machine Translation systems
- Domain classification service
- General purpose automatic classification toolbox based on transformers
- Field of Science (FoS) and Sustainable Development Goals (SDG) classification services
- Topic Modeling service
- Time Analysis service
- Graph Calculation and Analysis service
4) Development and deployment of IntelComp tools:
- Interactive Model Trainer
- STI Viewer
- Participation Portal
- Evaluation Workbench
- Auxiliary Tools: Data catalogue, Graph Explorer
5) Living Labs Implementation
- Refinement of the theoretical framework for the requirements and needs of each LL
- Operationalisation of IntelComp tools through the implementation of relevant use cases
- Diffusion of the project results and achievements to Public Administrations and STI policy stakeholders actively engaged in the use cases of the three Living Labs
A long list of different business scenarios has been gathered and iterated with partners for the future exploitation of the tools and the IntelComp platform as a whole. Project results have also been disseminated through scientific publications, policy briefs, project reports and a large number of LLs events targeted to relevant stakeholders.
From a technical perspective, IntelComp has developed novel Natural Language Processing/AI services for STI data analysis. It has developed an integrated platform allowing the joint exploitation of these services.
Some specific advancements beyond the state of the art are:
1) An enhanced, higher quality OpenAIRE research graph has been developed
2) Fine-tuned models for machine translation in the fields of AI and Cancer.
4) Algorithmic proposals:
- Zero-shot classification based on transformers
- Topic modeling: User-in-the-loop curation, hierarchical, federated
- SDG and FOS automatic classification
- Large-scale graph calculation and analysis methods
5) A graphical user interface to allow remote training of models
6) Scalable graph visualisation and exploration, exploiting also novel hierarchical graph representations
IntelComp has focused on engaging Public Administrations and policy stakeholders, and on working closely with them through the implementation of three Living Labs. At the end of the project, this list of engaged actors encompassed 168 individuals actively involved, including policy officers from the European Commission and the OECD, national ministries or funding agencies from countries like France, Greece, and Spain, and key figures within the STI policy stakeholders community in Greece (focusing on the AgriFood and Energy sectors) and Spain (from the AI sector). Six Public Administrations outside the consortium were actively involved as primary participants in the Living Labs.
The software developed by the project has been licensed under permissive FOSS licenses, and is available at the IntelComp profile at GitHub (https://github.com/IntelCompH2020). Further documentation can be found at the IntelComp community in zenodo (https://zenodo.org/communities/intelcomp).
Some specific advancements beyond the state of the art are:
1) An enhanced, higher quality OpenAIRE research graph has been developed
2) Fine-tuned models for machine translation in the fields of AI and Cancer.
4) Algorithmic proposals:
- Zero-shot classification based on transformers
- Topic modeling: User-in-the-loop curation, hierarchical, federated
- SDG and FOS automatic classification
- Large-scale graph calculation and analysis methods
5) A graphical user interface to allow remote training of models
6) Scalable graph visualisation and exploration, exploiting also novel hierarchical graph representations
IntelComp has focused on engaging Public Administrations and policy stakeholders, and on working closely with them through the implementation of three Living Labs. At the end of the project, this list of engaged actors encompassed 168 individuals actively involved, including policy officers from the European Commission and the OECD, national ministries or funding agencies from countries like France, Greece, and Spain, and key figures within the STI policy stakeholders community in Greece (focusing on the AgriFood and Energy sectors) and Spain (from the AI sector). Six Public Administrations outside the consortium were actively involved as primary participants in the Living Labs.
The software developed by the project has been licensed under permissive FOSS licenses, and is available at the IntelComp profile at GitHub (https://github.com/IntelCompH2020). Further documentation can be found at the IntelComp community in zenodo (https://zenodo.org/communities/intelcomp).