Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations

Informazioni relative al progetto

Data Synthesis Power

ID dell’accordo di sovvenzione: 792119

DOI

10.3030/792119

Progetto concluso il 31 Luglio 2019

Data della firma CE 26 Marzo 2018

Data di avvio 1 Settembre 2018

Data di completamento 31 Agosto 2020

Finanziato da

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Costo totale

€ 165 598,80

Contributo UE

€ 165 598,80

Coordinato da

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - Data Synthesis Power (Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations)

Periodo di rendicontazione: 2018-09-01 al 2020-08-31

Sintesi del contesto e degli obiettivi generali del progetto

Scientists, organizations, and governments are moving in the direction of sharing their data. This will lead to an unprecedented amount of available data that researchers could use as prior information in a Bayesian analysis. While Bayesian methods are a promising method for using the growing amount of available information to increase statistical power in research studies, there is a great need for developing guidelines for calibrating existing data to create accurate informative prior distributions. Bayesian methods that incorporate accurate prior information in the statistical analysis reduce the required sample size of the new study without decreasing the chance of detecting a true effect. However, prior data may be from a different population, measured using different instruments, and/or collected using different experimental procedures than those in the current study. Using the results from a prior study that differs from the current study in any of these respects as prior information in a Bayesian analysis can lead to biased results.

Several promising calibration methods have been proposed for linear regression analysis. In this project I proposed to extend and test these methods to models with third variables that reveal the conditions under which an effect exists (moderators), the mechanism through which one variable affects another (mediators), and variables that might explain an observed effect (confounders).

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

Due to the project being terminated early, the researcher completed only a chapter on the nature of exchangeability in social sciences studies and stared writing SAS code for a simulation study to examine the potential increase in statistical power with the use of exchangeable and non-exchangeable prior information.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

This project has been terminated early and the expected results were not fully achieved: unfortunately, 11 months is too short a time to make progress beyond the state of the field. One conference presentation was given in 2019.

What has been achieved in the 11 months: the sub-project on mediation analysis was started, and SAS code was written to examine different methods for downweighing prior information for mediation analysis. Progress beyond the current state of the art would have been made had the results from the simulation study been done and translated into guidelines for applied researchers. But when the project was terminated the code was still running (a five-month process).

Results that were not achieved due to early termination: no statistical methods for downweighing prior information for models with moderators and confounders were examined in simulation studies, and thus no guidelines and software were developed for these two statistical models.

The expected impact of the achieved aims of the project is a clearer understanding of when prior studies can be used in the statistical analysis without calibration and when calibration is necessary. Furthermore, the SAS code for the simulation study to evaluate resulting statistical properties of using exchangeable and non-exchangeable prior information in mediation analysis is currently running and the researcher will write a paper based on it. The expected impact of the paper are guidelines for researchers in the social sciences on when it is advisable to use existing studies in a Bayesian analysis.