Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations

Información del proyecto

Data Synthesis Power

Identificador del acuerdo de subvención: 792119

DOI

10.3030/792119

Proyecto finalizado el 31 Julio 2019

Fecha de la firma de la CE 26 Marzo 2018

Fecha de inicio 1 Septiembre 2018

Fecha de finalización 31 Agosto 2020

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 165 598,80

Aportación de la UE

€ 165 598,80

Coordinado por

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - Data Synthesis Power (Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations)

Período documentado: 2018-09-01 hasta 2020-08-31

Resumen del contexto y de los objetivos generales del proyecto

Scientists, organizations, and governments are moving in the direction of sharing their data. This will lead to an unprecedented amount of available data that researchers could use as prior information in a Bayesian analysis. While Bayesian methods are a promising method for using the growing amount of available information to increase statistical power in research studies, there is a great need for developing guidelines for calibrating existing data to create accurate informative prior distributions. Bayesian methods that incorporate accurate prior information in the statistical analysis reduce the required sample size of the new study without decreasing the chance of detecting a true effect. However, prior data may be from a different population, measured using different instruments, and/or collected using different experimental procedures than those in the current study. Using the results from a prior study that differs from the current study in any of these respects as prior information in a Bayesian analysis can lead to biased results.

Several promising calibration methods have been proposed for linear regression analysis. In this project I proposed to extend and test these methods to models with third variables that reveal the conditions under which an effect exists (moderators), the mechanism through which one variable affects another (mediators), and variables that might explain an observed effect (confounders).

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

Due to the project being terminated early, the researcher completed only a chapter on the nature of exchangeability in social sciences studies and stared writing SAS code for a simulation study to examine the potential increase in statistical power with the use of exchangeable and non-exchangeable prior information.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

This project has been terminated early and the expected results were not fully achieved: unfortunately, 11 months is too short a time to make progress beyond the state of the field. One conference presentation was given in 2019.

What has been achieved in the 11 months: the sub-project on mediation analysis was started, and SAS code was written to examine different methods for downweighing prior information for mediation analysis. Progress beyond the current state of the art would have been made had the results from the simulation study been done and translated into guidelines for applied researchers. But when the project was terminated the code was still running (a five-month process).

Results that were not achieved due to early termination: no statistical methods for downweighing prior information for models with moderators and confounders were examined in simulation studies, and thus no guidelines and software were developed for these two statistical models.

The expected impact of the achieved aims of the project is a clearer understanding of when prior studies can be used in the statistical analysis without calibration and when calibration is necessary. Furthermore, the SAS code for the simulation study to evaluate resulting statistical properties of using exchangeable and non-exchangeable prior information in mediation analysis is currently running and the researcher will write a paper based on it. The expected impact of the paper are guidelines for researchers in the social sciences on when it is advisable to use existing studies in a Bayesian analysis.