Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations

Informacje na temat projektu

Data Synthesis Power

Identyfikator umowy o grant: 792119

DOI

10.3030/792119

Realizacja projektu zakończyła się 31 Lipca 2019

Data podpisania przez KE 26 Marca 2018

Data rozpoczęcia 1 Września 2018

Data zakończenia 31 Sierpnia 2020

Finansowanie w ramach

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Koszt całkowity

€ 165 598,80

Wkład UE

€ 165 598,80

Koordynowany przez

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - Data Synthesis Power (Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations)

Okres sprawozdawczy: 2018-09-01 do 2020-08-31

Podsumowanie kontekstu i ogólnych celów projektu

Scientists, organizations, and governments are moving in the direction of sharing their data. This will lead to an unprecedented amount of available data that researchers could use as prior information in a Bayesian analysis. While Bayesian methods are a promising method for using the growing amount of available information to increase statistical power in research studies, there is a great need for developing guidelines for calibrating existing data to create accurate informative prior distributions. Bayesian methods that incorporate accurate prior information in the statistical analysis reduce the required sample size of the new study without decreasing the chance of detecting a true effect. However, prior data may be from a different population, measured using different instruments, and/or collected using different experimental procedures than those in the current study. Using the results from a prior study that differs from the current study in any of these respects as prior information in a Bayesian analysis can lead to biased results.

Several promising calibration methods have been proposed for linear regression analysis. In this project I proposed to extend and test these methods to models with third variables that reveal the conditions under which an effect exists (moderators), the mechanism through which one variable affects another (mediators), and variables that might explain an observed effect (confounders).

Prace wykonane od początku projektu do końca okresu sprawozdawczego oraz najważniejsze dotychczasowe rezultaty

Due to the project being terminated early, the researcher completed only a chapter on the nature of exchangeability in social sciences studies and stared writing SAS code for a simulation study to examine the potential increase in statistical power with the use of exchangeable and non-exchangeable prior information.

Innowacyjność oraz oczekiwany potencjalny wpływ (w tym dotychczasowe znaczenie społeczno-gospodarcze i szersze implikacje społeczne projektu)

This project has been terminated early and the expected results were not fully achieved: unfortunately, 11 months is too short a time to make progress beyond the state of the field. One conference presentation was given in 2019.

What has been achieved in the 11 months: the sub-project on mediation analysis was started, and SAS code was written to examine different methods for downweighing prior information for mediation analysis. Progress beyond the current state of the art would have been made had the results from the simulation study been done and translated into guidelines for applied researchers. But when the project was terminated the code was still running (a five-month process).

Results that were not achieved due to early termination: no statistical methods for downweighing prior information for models with moderators and confounders were examined in simulation studies, and thus no guidelines and software were developed for these two statistical models.

The expected impact of the achieved aims of the project is a clearer understanding of when prior studies can be used in the statistical analysis without calibration and when calibration is necessary. Furthermore, the SAS code for the simulation study to evaluate resulting statistical properties of using exchangeable and non-exchangeable prior information in mediation analysis is currently running and the researcher will write a paper based on it. The expected impact of the paper are guidelines for researchers in the social sciences on when it is advisable to use existing studies in a Bayesian analysis.