Final Report Summary - PSARPS (Practical statistical approaches for addressing replicability problems in life sciences)
The emerging issue of replicability in science was not a central issue when this research project was proposed, relying on an article from the magazine “New Yorker” as motivation. Over the six years duration of the project, the awareness to the lack of replication of discoveries has tremendously increased, and it is often referred to as the ‘replication crisis’ or the ‘reproducibility crisis,’ by the scientific establishment. Editorials have appeared in the leading general scientific journals: Science, Nature, Nature Methods and the Proceeding of the US National Academy of Science. Committees by learned societies have been created to assess the situation and offer remedies and organizations promoting Open Science were established. However most of the initial writing merely surfaced the existing problems and then came the emphasis transparency of experimental protocol, methods and data. We refer to these issues as the reproducibility of the original research plan, execution and analysis. While having a critical role, they are still not enough to statistically identify discoveries that are bound not be replicated in independent studies. In our research we have identified two such root causes statistical issues that hamper replicability. The first is that of selective inference, whehe the large number of potential findings, stemming mainly from the industrialization of the scientific process, are first screened to find the promising ones but that selection is later ignored. The second one is taking too optimistic point of view about the inherent variability in the process, such as those arising from random laboratories or studies being involved. In this research we offered new new general methodologies that can be used across science, as well as wide range of practical solutions to the particularities of many problems across many disciplines: medical research, pre-clinical research including research into mice behavior, brain research, epidemiology, genomics and experimental psychology. We demonstrated their use in particular studies and held educational activities, so that they can become known, available, and usable in the relevant fields. We envision that adopting widely their use in these fields will bring the false discovery rate down to the desirable level.