Open data: improving transparency, reproducibility and collaboration in science

Project Information

OPTIMISE

Grant agreement ID: 838237

DOI

10.3030/838237

Project closed

EC signature date 23 September 2019

Start date 1 January 2020

End date 31 December 2022

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 265 606,08

EU contribution

€ 265 606,08

Coordinated by

UNIVERSITE DE NEUCHATEL
Switzerland

Periodic Reporting for period 2 - OPTIMISE (Open data: improving transparency, reproducibility and collaboration in science)

Reporting period: 2022-01-01 to 2022-12-31

Summary of the context and overall objectives of the project

Open data involves making research data publicly available on an online database so they can be permanently stored, accessed, and (re)used by researchers and members of the public. Open data are a fundamental requirement of reproducible research and benefit society by speeding up scientific discoveries. Without access to data underlying published studies, the scientific community is hard pressed to validate and build on scientific discoveries. For this reason, many publishers and funding agencies have begun mandating that researchers publicly archive their datasets as a condition of publication or funding. However, open data policies remain a topic of debate within the scientific community and the adoption of good data sharing practices is slow in many disciplines. For example, some researchers feel strong ownership over their data and fear that potential individual benefits (e.g. increased citation rate, opportunities for co-authorship and new collaborations) will not compensate for any future publications lost by relinquishing priority of access to the data they collected – the fear of being ‘scooped’. At present, apprehensive authors can avoid publishing in journals with an open data policy or share their data in ways that make them difficult or impossible to re-use. There are also concerns about publicly sharing sensitive data that contain private information about human participants and/or the location of endangered species. Nevertheless, responsible open data practices hold the promise of greater transparency and reproducibility in science, with the potential to drastically accelerate research. However, further evidence is needed to verify the efficacy of open data at achieving these aims and assuage beliefs that open data can impede career advancement. The objectives of this action were to examine: (1) the effectiveness of open data at improving the effectiveness and impact of research in environmental science, and (2) the barriers and motivations affecting the decisions of researchers to adopt good open data practices. The findings suggest that improvements to the archiving quality of open data in ecology and evolution has been slow over the last decade, with a lack of researcher training and education identified as one of the main factors affecting poor data archiving practices. Editorial open data policies are largely successful at getting researchers to share their research data but have little to no impact on the completeness and reusability of open datasets. Finally, there is no evidence that good data sharing practices improve transparency in how researchers report their methods and results or that mandatory open data policies by journals increase error correction in ecology and evolution.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The work performed during the 3-year action led to 10 published papers and 2 manuscripts.

Several projects were conducted to assess the benefits and challenges of open data in environmental science. Tools and recommendations were developed to overcome barriers to data sharing and improve its benefits. One study examined the Government of Canada’s initiatives promoting open data, and summarized research data management (RDM) challenges, plans to modernize RDM, and best practices for data discoverability, access, and reuse. Another study outlined risks associated with publicly sharing sensitive data from animal tracking studies and proposed a framework for making these data ‘as open as possible and as closed as necessary’.. Another study outlined sources of wasted research resources in conservation science and highlighted how data rescue and reuse, open science practices, and knowledge co-creation can help improve the effectiveness and impact of conservation science. Finally, an analysis examined how open science practices can contribute to bridging the knowledge-action gap in conservation science and practice.

Other projects examined the barriers and motivations affecting decisions by researchers to engage in data sharing. In a first study, we presented the results of a survey examining data sharing practices and reported costs and benefits of sharing open data among Canadian faculty members in ecology and evolution. In a second study, we identified slow improvements to the archiving quality of open datasets shared by researchers in ecology and evolution between 2012-2019 and found that researcher training could be the single most important factor affecting data archiving quality. In a third study, we found that cooperative and non-cooperative researchers share open data of similar quality, suggesting that a lack of training in research data management impedes high-quality data sharing by cooperative scientists. In a fourth study, we defined the concept of data rescue and developed a practical framework for saving environmental data from extinction. In a fifth study, we examined how good data sharing practices and meaningful interdisciplinary collaboration can lead to greater consensus building in experimental biology, reviewing data and code sharing practices in over 1,500 empirical studies.

Two additional projects empirically assessed suggested benefits of open data. One found no evidence that mandatory open data policies increase error correction in ecology and evolution. The other found no evidence that sharing high quality open data is linked to greater transparency in how a study’s methods and results are reported.

The action’s results were communicated via invited talks, conference presentations, workshops, social media, and news articles. Insights from the action helped shape science policy in Canada and in Switzerland via my involvement in evaluation panels, working groups, policy committees, sounding boards, and learned societies.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Metascience uses the tools of science and state-of-the-art methods to examine strengths and weaknesses in research practices, with the aim of improving transparency, reproducibility, and equity in science. While metascience originated in the fields of biomedicine and psychology, comparatively little work has been done in environmental science. By focusing on the fields of ecology, evolution, and conservation biology, the action contributed to bridging this gap. The action also provided some of the first empirical data on costs and benefits experienced by researchers as well as factors that are predictive of good data sharing practices. Previous metascience on potential costs and benefits of open data had focused on the attitudes and fears of researchers rather than actual costs and benefits. The action's results suggest that researcher training and education in research data management are key to improving open data. These findings will help inform science policy, improve training and incentive structures in academia, and change researcher behaviours. The practical recommendations formulated to improve data sharing practices will increase the transparency, reproducibility, and (re)usability of research outputs in environmental science, benefiting both science and society. The action focused on open data in environmental science, but data sharing is transdisciplinary and the insights gained are relevant to researchers and policymakers across fields. The resulting papers have been cited in NGO and governments briefs, and by researchers in psychology, geology, medecine, sports and exercise science, and education.

Sharing FAIR data is challenging and requires training, technical support, and incentives.