Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations

Informations projet

Data Synthesis Power

N° de convention de subvention: 792119

DOI

10.3030/792119

Projet clôturé le 31 Juillet 2019

Date de signature de la CE 26 Mars 2018

Date de début 1 Septembre 2018

Date de fin 31 Août 2020

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 165 598,80

Contribution de l’UE

€ 165 598,80

Coordonné par

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - Data Synthesis Power (Reducing required sample size of new studies by effectively using historical data: new methods for evaluating mechanisms, conditions, and alternative explanations)

Période du rapport: 2018-09-01 au 2020-08-31

Résumé du contexte et des objectifs généraux du projet

Scientists, organizations, and governments are moving in the direction of sharing their data. This will lead to an unprecedented amount of available data that researchers could use as prior information in a Bayesian analysis. While Bayesian methods are a promising method for using the growing amount of available information to increase statistical power in research studies, there is a great need for developing guidelines for calibrating existing data to create accurate informative prior distributions. Bayesian methods that incorporate accurate prior information in the statistical analysis reduce the required sample size of the new study without decreasing the chance of detecting a true effect. However, prior data may be from a different population, measured using different instruments, and/or collected using different experimental procedures than those in the current study. Using the results from a prior study that differs from the current study in any of these respects as prior information in a Bayesian analysis can lead to biased results.

Several promising calibration methods have been proposed for linear regression analysis. In this project I proposed to extend and test these methods to models with third variables that reveal the conditions under which an effect exists (moderators), the mechanism through which one variable affects another (mediators), and variables that might explain an observed effect (confounders).

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

Due to the project being terminated early, the researcher completed only a chapter on the nature of exchangeability in social sciences studies and stared writing SAS code for a simulation study to examine the potential increase in statistical power with the use of exchangeable and non-exchangeable prior information.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

This project has been terminated early and the expected results were not fully achieved: unfortunately, 11 months is too short a time to make progress beyond the state of the field. One conference presentation was given in 2019.

What has been achieved in the 11 months: the sub-project on mediation analysis was started, and SAS code was written to examine different methods for downweighing prior information for mediation analysis. Progress beyond the current state of the art would have been made had the results from the simulation study been done and translated into guidelines for applied researchers. But when the project was terminated the code was still running (a five-month process).

Results that were not achieved due to early termination: no statistical methods for downweighing prior information for models with moderators and confounders were examined in simulation studies, and thus no guidelines and software were developed for these two statistical models.

The expected impact of the achieved aims of the project is a clearer understanding of when prior studies can be used in the statistical analysis without calibration and when calibration is necessary. Furthermore, the SAS code for the simulation study to evaluate resulting statistical properties of using exchangeable and non-exchangeable prior information in mediation analysis is currently running and the researcher will write a paper based on it. The expected impact of the paper are guidelines for researchers in the social sciences on when it is advisable to use existing studies in a Bayesian analysis.