Periodic Reporting for period 1 - IMPACT-SC5 (ASSESSING THE IMPACT PATHWAYS OF IA/RIA SC5 PROJECTS THROUGH THE USE OF PORTFOLIO ANALYSIS)
Période du rapport: 2019-11-01 au 2021-06-30
The Horizon 2020 Framework Programme for Research and Innovation was built around three pillars. The third one, dedicated to ‘Societal Challenges’, made a significant contribution to the paradigmatic change in Research and Innovation Policy according to which the pursued objectives were not only to increase economic competitiveness or improve the functioning of innovation systems, but also to help solve important socio-economic challenges. More specifically, with the overarching objective of meeting the needs of a growing global population within the sustainable limits of the planet's natural resources and eco-systems, the Societal Challenge 5 (SC5) “Climate action, environment, resource efficiency and raw materials”, aimed to achieve a resource and water – efficient and climate change resilient economy and society, the protection and sustainable management of natural resources and ecosystems, and a sustainable supply and use of raw materials.
In accordance with the above, the corresponding work programme 2014-2015 focused on investing in innovation for a green economy; addressing the gaps in the knowledge base needed to understand the changes occurring in the environment; identifying the policies, methods and tools that would most effectively tackle the abovementioned challenges; and, supporting innovators and businesses to bring green solutions to the market. Waste and water were selected as particular priorities, on the grounds of their substantial potential for business opportunities and job creation while tackling important resource efficiency challenges.
In total, 148 projects were selected for funding during the period 2014 – 2015 , out of which 87 were Research and Innovation Actions (RIAs) and Innovation Actions (IAs).
The transformation of innovation systems, which the projects granted by the SC5 Work Programme 2014-2015 intended to contribute to, necessitated a clear direction given to policy and research efforts, a careful consideration for the societal acceptance and market uptake of the solutions to be developed, and, finally, a ‘reflexive’ approach on the efforts that policymakers should make to monitor and measure the impacts of the funded actions.
Within this context, the IMPACT-SC5 project - “Assessing the Impact Pathways of IA/RIA SC5 Projects through the Use of Portfolio Analysis” aimed at designing and implementing an appropriate methodology based on the “Key Impact Pathways” time-sensitive approach to measure and explore the achievements of the 87 projects granted by the SC5 Work Programme 2014-2015, and their scientific, economic and societal impacts. For capturing comprehensively those impacts, the IMPACT-SC5 project adopts a multi-level approach and considers the impacts on: (1) the micro/project level; (2) meso/ the portfolios of projects level; and (3) the macro /programme level.
Meaningful policy recommendations for future programme actions and for future evaluation exercises based on the experience of implementing the IMPACT-SC5 project are also provided.
In accordance with the above, the corresponding work programme 2014-2015 focused on investing in innovation for a green economy; addressing the gaps in the knowledge base needed to understand the changes occurring in the environment; identifying the policies, methods and tools that would most effectively tackle the abovementioned challenges; and, supporting innovators and businesses to bring green solutions to the market. Waste and water were selected as particular priorities, on the grounds of their substantial potential for business opportunities and job creation while tackling important resource efficiency challenges.
In total, 148 projects were selected for funding during the period 2014 – 2015 , out of which 87 were Research and Innovation Actions (RIAs) and Innovation Actions (IAs).
The transformation of innovation systems, which the projects granted by the SC5 Work Programme 2014-2015 intended to contribute to, necessitated a clear direction given to policy and research efforts, a careful consideration for the societal acceptance and market uptake of the solutions to be developed, and, finally, a ‘reflexive’ approach on the efforts that policymakers should make to monitor and measure the impacts of the funded actions.
Within this context, the IMPACT-SC5 project - “Assessing the Impact Pathways of IA/RIA SC5 Projects through the Use of Portfolio Analysis” aimed at designing and implementing an appropriate methodology based on the “Key Impact Pathways” time-sensitive approach to measure and explore the achievements of the 87 projects granted by the SC5 Work Programme 2014-2015, and their scientific, economic and societal impacts. For capturing comprehensively those impacts, the IMPACT-SC5 project adopts a multi-level approach and considers the impacts on: (1) the micro/project level; (2) meso/ the portfolios of projects level; and (3) the macro /programme level.
Meaningful policy recommendations for future programme actions and for future evaluation exercises based on the experience of implementing the IMPACT-SC5 project are also provided.
IMPACT-SC5 kick-started by designing a robust evaluation framework containing the key concepts of the evaluation while incorporating a tailored conceptual intervention logic oriented to define the causal linkages between the elements of the evaluation. To accompany the evaluation framework, a set of short-, medium- and long-term indicators was constructed so as to monitor the performance of the projects towards their objectives. In total, 58 individual indicators have been developed across 27 Evaluation (Indicator) fiches.
With the evaluation framework in place, an extensive literature review exercise was conducted to collect existing information on the impacts of the projects under investigation through the use of openly available sources (such as CORDIS, projects’ webpages and social media accounts, public deliverables, newsletters, to name a few). On top of that, an individualised online survey complemented by a number of semi-structured interviews were conducted to collect quantitative and qualitative data on the scientific, societal and environmental, as well as economic level impacts of the SC5 projects. For each one of the 87 projects under scrutiny a short, yet concise, individual project fiche was developed containing all the data collected through the literature review analysis, online survey and interviews.
A portfolio impact assessment exercise was then implemented to complement the project-level analysis, measuring the collective impacts that projects have made together. As such, an impact pathways analysis of 5 aggregated groupings (portfolios) was performed so as not only to measure the performance of the groupings but also to draw meaningful policy-relevant conclusions. Five (5) thematic policy co-creation workshops were organised to discuss the results of the portfolio analysis and along with the participants to collectively identify how policy on national and European level can facilitate the enhancement of SC5 impacts’ diffusion and uptake across the EU.
With that in mind, an Actionable Knowledge Toolkit was developed translating the experience drawn from the implementation of the IMPACT-SC5 project into understandable and usable (actionable) adding-value content, tailored to policymakers and evaluation practitioners.
In parallel, a tailored dissemination and communication strategy was implemented to promote the results of IMPACT-SC5, while also fostering participation in project’s workshops and events.
With the evaluation framework in place, an extensive literature review exercise was conducted to collect existing information on the impacts of the projects under investigation through the use of openly available sources (such as CORDIS, projects’ webpages and social media accounts, public deliverables, newsletters, to name a few). On top of that, an individualised online survey complemented by a number of semi-structured interviews were conducted to collect quantitative and qualitative data on the scientific, societal and environmental, as well as economic level impacts of the SC5 projects. For each one of the 87 projects under scrutiny a short, yet concise, individual project fiche was developed containing all the data collected through the literature review analysis, online survey and interviews.
A portfolio impact assessment exercise was then implemented to complement the project-level analysis, measuring the collective impacts that projects have made together. As such, an impact pathways analysis of 5 aggregated groupings (portfolios) was performed so as not only to measure the performance of the groupings but also to draw meaningful policy-relevant conclusions. Five (5) thematic policy co-creation workshops were organised to discuss the results of the portfolio analysis and along with the participants to collectively identify how policy on national and European level can facilitate the enhancement of SC5 impacts’ diffusion and uptake across the EU.
With that in mind, an Actionable Knowledge Toolkit was developed translating the experience drawn from the implementation of the IMPACT-SC5 project into understandable and usable (actionable) adding-value content, tailored to policymakers and evaluation practitioners.
In parallel, a tailored dissemination and communication strategy was implemented to promote the results of IMPACT-SC5, while also fostering participation in project’s workshops and events.
IMPACT-SC5 was an explorative impact assessment study that tested and advanced the impact pathways approach by conducting a project portfolio analysis of 87 RIAs and IAs funded under the SC5 2014-2015 Work Programme. The main results and recommendations of IMPACT-SC5 were disseminated to and discussed with the research performers (SC5 project coordinators & partners), national and European stakeholders (NCPs, European stakeholder organisations) and the Commission (DG Research & Innovation, REA) through a dedicated online policy workshop series. These workshops took place in spring 2021 and addressed the fulfilment of impact pathways in five thematic areas of the SC5 2014-2015 Work Programme: 1) Climate Change, 2) Environment, Ecosystems, Biodiversity, 3) Raw Materials, 4) Waste and 5) Water.
The main recommendations of IMPACT-SC5 for future programme actions concerned, among others, how the impact pathways approach should inform the full policy cycle, availability of training materials on impact pathways, and accessibility of project legacy (results and data of SC5 projects) via online platforms. Regarding the main recommendations for future evaluation exercises, the issues of improved impact terminology, short-listing of measurable impact indicators and overcoming the GDPR and privacy issues were highlighted by IMPACT-SC5, among others.
The main recommendations of IMPACT-SC5 for future programme actions concerned, among others, how the impact pathways approach should inform the full policy cycle, availability of training materials on impact pathways, and accessibility of project legacy (results and data of SC5 projects) via online platforms. Regarding the main recommendations for future evaluation exercises, the issues of improved impact terminology, short-listing of measurable impact indicators and overcoming the GDPR and privacy issues were highlighted by IMPACT-SC5, among others.