Periodic Reporting for period 2 - OSIRIS (Open Science to Increase Reproducibility in Science)
Período documentado: 2024-01-01 hasta 2025-06-30
Reproducibility is essential for Research and Innovation (R&I) as it validates or corrects study outcomes, leading to higher quality, more reliable, and cost-effective research. Yet, only a fraction of studies can be reproduced due to constant innovation demands, publication pressure, poor reporting, and assessment systems focused on quantity over quality. With limited incentives, the burden currently falls on researchers, and proposed improvements remain untested. To restore trust in science, a cultural and systemic shift is needed. OSIRIS aims to drive this by identifying drivers, testing evidence-based solutions, engaging stakeholders, and embedding reproducibility in research design, with the ambition of making it widely accepted, practiced, and recognized globally by 2026.
Objectives:
-To understand the underlying drivers and effective interventions that increase reproducibility at funding, publishing, university, and researcher-level using systematic literature review, evidence mapping, policy audits, and interviews and focus group discussions with stakeholders. Results will be distributed through an open knowledge base and Open Access (OA) publications to optimally reach global academia.
-To develop and test effective, evidence-based solutions for the reproducibility crisis across various stakeholders in policy and research practice by utilising well-controlled Randomised Controlled Trials (RCTs) rather than mere pilots, develop dashboards of indicators of reproducible research practices, and providing funders, publishers, researchers, and peer reviewers with guidance for judging reproducibility.
-To embed reproducibility in the strategy and design of research projects by informing researchers and convincing funders and journals to include measures and preconditions on reproducibility in their assessment of project proposals and articles.
-To create a collaborative community of stakeholders that will aid in educating and implementing better reproducible research practice using our results to create guidelines and training on how researchers can embed reproducibility in the design of their research and disseminate these widely, thereby increase the reproducibility of their scientific research. Additionally, we will perform quality audits at project and output level to test these novel practices.
Our interdisciplinary consortium combines expertise in Open Science, reproducibility, empirical research, and RCT interventions with strong stakeholder engagement to deliver evidence-based, widely supported solutions. Leveraging our broad network of reproducibility leaders, journals, funders, and policy officers, we will ensure broad uptake of OSIRIS results across scientific fields and institutions.
Objectives:
-To understand the underlying drivers and effective interventions that increase reproducibility at funding, publishing, university, and researcher-level using systematic literature review, evidence mapping, policy audits, and interviews and focus group discussions with stakeholders. Results will be distributed through an open knowledge base and Open Access (OA) publications to optimally reach global academia.
-To develop and test effective, evidence-based solutions for the reproducibility crisis across various stakeholders in policy and research practice by utilising well-controlled Randomised Controlled Trials (RCTs) rather than mere pilots, develop dashboards of indicators of reproducible research practices, and providing funders, publishers, researchers, and peer reviewers with guidance for judging reproducibility.
-To embed reproducibility in the strategy and design of research projects by informing researchers and convincing funders and journals to include measures and preconditions on reproducibility in their assessment of project proposals and articles.
-To create a collaborative community of stakeholders that will aid in educating and implementing better reproducible research practice using our results to create guidelines and training on how researchers can embed reproducibility in the design of their research and disseminate these widely, thereby increase the reproducibility of their scientific research. Additionally, we will perform quality audits at project and output level to test these novel practices.
Our interdisciplinary consortium combines expertise in Open Science, reproducibility, empirical research, and RCT interventions with strong stakeholder engagement to deliver evidence-based, widely supported solutions. Leveraging our broad network of reproducibility leaders, journals, funders, and policy officers, we will ensure broad uptake of OSIRIS results across scientific fields and institutions.
During RP2, we have completed the following deliverables:
D2.1. Protocols (for the studies in WP2) published on OSF
D3.1. OSIRIS networks for computational and methods reproducibility
D3.2. Observatory dashboard
D2.2 OA paper(s) on drivers, barriers and facilitators for reproducibility of research
D2.3 Report on policy audit with model policies
D2.4 OA paper “Ten tales of reproducibility”
D6.6 RP2 Updated Dissemination and Communication Plan
D6.9 Policy Brief RP2
We have also finalised the scoping review (WP2), the barriers and facilitators study (WP2), engaged in citizen participation (WP3) and finalised the policy audit (WP2).
We have an ongoing fruitful collaboration with two other EU-funded projects: TIER2 and iRISE. This allows for the exchange of knowledge among the projects and avoids repetition of the work performed. Additionally, collaboration between the projects is a ‘multiplayer’, where we hope to extend our goals and create better and more extensive outcomes.
Additionally, WP6 has successfully advanced the communication and dissemination strategy of the OSIRIS project, effectively meeting its core objectives. The Dissemination and Communication Plan (DCP) has been operationalised and dynamically implemented, resulting in widespread awareness of OSIRIS activities, outputs, and impact.
D2.1. Protocols (for the studies in WP2) published on OSF
D3.1. OSIRIS networks for computational and methods reproducibility
D3.2. Observatory dashboard
D2.2 OA paper(s) on drivers, barriers and facilitators for reproducibility of research
D2.3 Report on policy audit with model policies
D2.4 OA paper “Ten tales of reproducibility”
D6.6 RP2 Updated Dissemination and Communication Plan
D6.9 Policy Brief RP2
We have also finalised the scoping review (WP2), the barriers and facilitators study (WP2), engaged in citizen participation (WP3) and finalised the policy audit (WP2).
We have an ongoing fruitful collaboration with two other EU-funded projects: TIER2 and iRISE. This allows for the exchange of knowledge among the projects and avoids repetition of the work performed. Additionally, collaboration between the projects is a ‘multiplayer’, where we hope to extend our goals and create better and more extensive outcomes.
Additionally, WP6 has successfully advanced the communication and dissemination strategy of the OSIRIS project, effectively meeting its core objectives. The Dissemination and Communication Plan (DCP) has been operationalised and dynamically implemented, resulting in widespread awareness of OSIRIS activities, outputs, and impact.
Work Packages (WP2–6) addressed key drivers, barriers, and solutions to improve reproducibility across the research system. WP2 identified limited rigorous evidence for Open Science interventions, low adoption of reproducibility policies beyond data sharing, and highlighted researcher motivations and real-world cases through the Ten Tales of Reproducibility. WP3 focused on developing an Open Science Checklist and establishing a researcher network to test and apply it. WP4 tested the checklist in both journal peer review (402 manuscripts at The BMJ and BMJ Open) and funding proposal review (101 grants), assessing its impact on reproducibility prediction. WP5 initiated training development through a cross-disciplinary researcher roundtable to shape a prototype training resource. WP6 advanced OSIRIS’s communication and dissemination strategy, successfully implementing the Dissemination and Communication Plan and raising widespread awareness of project activities and impact.