Periodic Reporting for period 3 - SUPER_MoRRI (Scientific Understanding and Provision of an Enhanced and Robust Monitoring system for RRI)
Reporting period: 2022-01-01 to 2023-12-31
Across Europe, research performing and research funding organisations (RPOs and RFOs) increasingly appreciate the need for a more dynamic governance and a better societal integration of research and innovation (R&I). Internal drivers of change (such as the digitalization of science and AI) and political will to better align with societal needs are bringing about aspirational policies and processes of transformation of the R&I system, including those of RRI – Responsible Research and Innovation.
In order for the aspirations of RRI to be realised, robust tools are needed for R&I policy and practice. The MoRRI project (2014-2018) conceptualised and implemented the first RRI monitoring system in Europe. The SUPER MoRRI project built upon and continued the work of MoRRI, ensuring sustained data collection, curation, further assessment and refinement of the MoRRI indicators. EU-27 data was complemented by monitoring data from selected non-EU countries. Furthermore, SUPER MoRRI went well beyond the technical efforts of MoRRI and developed a more complete scientific understanding of the complex and diverse relationships between RRI policies and practices and their societal, democratic, economic and scientific benefits.
To meet these ambitious research challenges, the original scientific partners of MoRRI have included complementary RRI expertise to form the truly interdisciplinary SUPER MoRRI consortium. In line with the concept of RRI itself, our learning processes were also enhanced by participatory co-creation practices with stakeholders.
To ensure a high degree of accessibility, the key outputs of SUPER MoRRI are made available on the PROMISE portal (Platform for the Support of Responsibility and Openness and their Monitoring in Innovation and Science Ecosystems - www.promise4era.eu). Specifically, PROMISE also provides an easy-to-use and easy-to-customize online dashboard system serving a variety of uses and end-users.
In order for the aspirations of RRI to be realised, robust tools are needed for R&I policy and practice. The MoRRI project (2014-2018) conceptualised and implemented the first RRI monitoring system in Europe. The SUPER MoRRI project built upon and continued the work of MoRRI, ensuring sustained data collection, curation, further assessment and refinement of the MoRRI indicators. EU-27 data was complemented by monitoring data from selected non-EU countries. Furthermore, SUPER MoRRI went well beyond the technical efforts of MoRRI and developed a more complete scientific understanding of the complex and diverse relationships between RRI policies and practices and their societal, democratic, economic and scientific benefits.
To meet these ambitious research challenges, the original scientific partners of MoRRI have included complementary RRI expertise to form the truly interdisciplinary SUPER MoRRI consortium. In line with the concept of RRI itself, our learning processes were also enhanced by participatory co-creation practices with stakeholders.
To ensure a high degree of accessibility, the key outputs of SUPER MoRRI are made available on the PROMISE portal (Platform for the Support of Responsibility and Openness and their Monitoring in Innovation and Science Ecosystems - www.promise4era.eu). Specifically, PROMISE also provides an easy-to-use and easy-to-customize online dashboard system serving a variety of uses and end-users.
The key objectives that the SUPER MoRRI project addressed were extremely challenging both scientifically as well as with regard to the potential implications of the project’s results. Scientifically, because the measurement of scientific, economic, societal and political impacts of policies and measures aimed at increasing responsible practices in European research and innovation activities requires the thorough revision of existing and the development of novel conceptual approaches to monitoring and indicators. To this end, existing indicators and previous experiences with data collection were critically examined and put on the test bed, important conceptual foundations for the development of the RRI monitoring framework were developed, and a plan for the case research was developed (see deliverables D1.2 D2.1 and 5.1). As any indicator system is performative and likely to generate unintended effects, the SUPER MoRRI consortium was particularly committed to responsible conduct in the process of developing the monitoring framework.
A key element of the SUPER MoRRI project was the exchange with and involvement of different networks of stakeholders. These different but interrelated networks functioned as sounding boards, critical friends and, most importantly, were involved in the processes of co-constructing and testing the monitoring framework and of the data collection procedures, the online dashboard, and the self-assessment tool (SAT).
Based on the strategic foundations and enriched by feedbacks from the stakeholders, SUPER MoRRI set in motion a number of data collection streams, case research and development work: The data collection for the monitoring framework included selected secondary data (including Eurobarometer data), the operationalization and implementation of the RPO and the RFO studies as well as a survey of European researchers. The project also conducted an ambitious case study research programme with the aim to improve the understanding of how RRI activities generate impact on research and innovation practices and institutional structures. The six research cases covered different dimensions of RRI, reaching from the role of civil society organisations (CSOs) at the science-society interface and ethics in AI to a study of public value research careers. And with the aim of providing practical support to research performing organizations (RPOs) interested in reaching higher levels of responsibility, SUPER MoRRI developed an online self-assessment tool (SAT).
Of the numerous outputs generated over the course of the project, the following deserve particular emphasis:
- the three RRI Monitoring Reports (D2.2 D2.3 and D2.5)
- the case research (D5.2 D5.3 and D5.4)
- the international perspectives (D4.2) and recommendations on RRI (D4.3).
Most importantly, an online portal was established (PROMISE, available at www.promise4era.eu) making available key project results and data. PROMISE includes an online indicator dashboard, providing dynamic and customizable visualizations, access to primary data from the RFO and RPO studies, the researchers survey and the case studies. The self-assessment tool (SAT) can also be accessed via the portal.
The final report (D1.4) provides a concise and brief overview of the main results of SUPER MoRRI, including key policy recommendations.
A key element of the SUPER MoRRI project was the exchange with and involvement of different networks of stakeholders. These different but interrelated networks functioned as sounding boards, critical friends and, most importantly, were involved in the processes of co-constructing and testing the monitoring framework and of the data collection procedures, the online dashboard, and the self-assessment tool (SAT).
Based on the strategic foundations and enriched by feedbacks from the stakeholders, SUPER MoRRI set in motion a number of data collection streams, case research and development work: The data collection for the monitoring framework included selected secondary data (including Eurobarometer data), the operationalization and implementation of the RPO and the RFO studies as well as a survey of European researchers. The project also conducted an ambitious case study research programme with the aim to improve the understanding of how RRI activities generate impact on research and innovation practices and institutional structures. The six research cases covered different dimensions of RRI, reaching from the role of civil society organisations (CSOs) at the science-society interface and ethics in AI to a study of public value research careers. And with the aim of providing practical support to research performing organizations (RPOs) interested in reaching higher levels of responsibility, SUPER MoRRI developed an online self-assessment tool (SAT).
Of the numerous outputs generated over the course of the project, the following deserve particular emphasis:
- the three RRI Monitoring Reports (D2.2 D2.3 and D2.5)
- the case research (D5.2 D5.3 and D5.4)
- the international perspectives (D4.2) and recommendations on RRI (D4.3).
Most importantly, an online portal was established (PROMISE, available at www.promise4era.eu) making available key project results and data. PROMISE includes an online indicator dashboard, providing dynamic and customizable visualizations, access to primary data from the RFO and RPO studies, the researchers survey and the case studies. The self-assessment tool (SAT) can also be accessed via the portal.
The final report (D1.4) provides a concise and brief overview of the main results of SUPER MoRRI, including key policy recommendations.
SUPER MoRRI aimed at furthering inclusivity, collaboration and transparency in R&I systems by improving the understanding of the evolution and the development of benefits of RRI.
To provide orientation, SUPER MoRRI developed two key guiding principles for the project and beyond, which we suggest to be broadly applied and taken into account whenever metrics and indicators are developed and used:
- Credible contextualisation: There are no universal context-free indicators; indicators should be developed in ways relevant and meaningful in specific use contexts and should pass through a co-creation phase with potential users.
- Responsible quantification: Data and information provided to users as a resource is prepared, presented, and made interpretable in appropriate ways.
With regard to monitoring and evaluation in general, the fact that numerous initiatives promoting a more responsible use of indicators and metrics mushroomed in the recent past – such as the Leiden Manifesto, Responsible Metrics for Open Science, DORA, CoARA and others – shows the growing support for this type of change. But also the broader shifts in R&I policy, reflected by more directionality to address societal challenges and new approaches such as mission-oriented innovation policy, actually call for more, not less anticipation, responsiveness, ethics and integrity, participation and co-creation in view of the far-reaching changes for everyday lives these transformative types of research and innovation entail. Without careful and prudent governance and open and responsible research and innovation practices, intensified controversies and opposition to the needed transitions are likely. We are convinced that the findings and outputs of the SUPER MoRRI project can contribute to the development of renewed R&I governance and practices.
Key sources for further information:
- PROMISE-Portal: www.promise4era.eu
- Repository for project deliverables and publications: https://zenodo.org/communities/supermorri/records
To provide orientation, SUPER MoRRI developed two key guiding principles for the project and beyond, which we suggest to be broadly applied and taken into account whenever metrics and indicators are developed and used:
- Credible contextualisation: There are no universal context-free indicators; indicators should be developed in ways relevant and meaningful in specific use contexts and should pass through a co-creation phase with potential users.
- Responsible quantification: Data and information provided to users as a resource is prepared, presented, and made interpretable in appropriate ways.
With regard to monitoring and evaluation in general, the fact that numerous initiatives promoting a more responsible use of indicators and metrics mushroomed in the recent past – such as the Leiden Manifesto, Responsible Metrics for Open Science, DORA, CoARA and others – shows the growing support for this type of change. But also the broader shifts in R&I policy, reflected by more directionality to address societal challenges and new approaches such as mission-oriented innovation policy, actually call for more, not less anticipation, responsiveness, ethics and integrity, participation and co-creation in view of the far-reaching changes for everyday lives these transformative types of research and innovation entail. Without careful and prudent governance and open and responsible research and innovation practices, intensified controversies and opposition to the needed transitions are likely. We are convinced that the findings and outputs of the SUPER MoRRI project can contribute to the development of renewed R&I governance and practices.
Key sources for further information:
- PROMISE-Portal: www.promise4era.eu
- Repository for project deliverables and publications: https://zenodo.org/communities/supermorri/records