Periodic Reporting for period 4 - ENHANCEDQMMR (Enhanced Qualitative and Multi-Method Research in Political Science)
Período documentado: 2019-12-01 hasta 2020-04-30
Social scientists need methods, techniques and tools for answering their research question and contributing to knowledge generation and inform policy-making. Proper methods are not all that there is to science, but science requires the right tools that must be correctly applied to produce correct findings. This project focuses on a set of methods comprising qualitative techniques such as process tracing and Qualitative Comparative Analysis (QCA) and multimethod research (MMR) combining different methods. The project aims to improve these methods and their application in multiple ways and to enable empirical researchers to be more confident in the accuracy and reproducibility of their findings.
In the first half, the project focused on Qualitative Comparative Analysis (QCA). The project developed and used simulations for contributing to the rigor of QCA. This includes a better understanding of how susceptible QCA is to produce correct results even if irrelevant conditions are included in the initial model. Another achievement is the development of a technique for estimating the power of a QCA study, which means to estimate the probability of correctly rejecting a false null hypothesis. Further projects include the analysis of clustered data with QCA; the presentation and implementation of robustness tests with QCA; an analysis of publication bias in empirical QCA research and how to diminish it; the choice between set types in empirical QCA research; the substantive interpretation of QCA results using informative variable metrics. For four of these subprojects, there are or will be R packages that empirical researchers can use in their own work.
In work package two, we ran a survey experiment that aims to understand better how social scientists decide about the research design that they implement and what inferences they make conditional on the nature and amount of data that they see. We further wrote a paper that discusses theories of causation, causal inference and social science methods from a comprehensive perspective and performed a novel and innovative survey of researchers asking them for their view of social science methods and causal inference.
In work package three, we performed simulations comparing QCA results with regression results for a wide number of different set-relational data-generating processes. The simulations allows us to understand better when and under what conditions QCA derives the correct solutions from the data and when regression produces results that mimic the data-generating process.
In work package four, we develop Bayesian nested analysis as an alternative to the conventional frequentist version. An R package will be available for the implementation of Bayesian nested analysis in empirical research.
In work package two, we ran a survey experiment that aims to understand better how social scientists decide about the research design that they implement and what inferences they make conditional on the nature and amount of data that they see. We further wrote a paper that discusses theories of causation, causal inference and social science methods from a comprehensive perspective and performed a novel and innovative survey of researchers asking them for their view of social science methods and causal inference.
In work package three, we performed simulations comparing QCA results with regression results for a wide number of different set-relational data-generating processes. The simulations allows us to understand better when and under what conditions QCA derives the correct solutions from the data and when regression produces results that mimic the data-generating process.
In work package four, we develop Bayesian nested analysis as an alternative to the conventional frequentist version. An R package will be available for the implementation of Bayesian nested analysis in empirical research.
The developments so far contribute to the state of the art because they develop simulations, which were still relatively new to the field of QCA, for improving QCA. In the second half of the project, we designed and started an experiment allowing us to better understand what influences the conclusions that empirical researchers based on data. This helps sensitizing researchers for potential fallacies when interpreting data for making causal inferences. With regard to QCA and multimethod research, we will work on multiple ends to strengthen QCA, including for example the development of safeguards against publication bias in QCA research and the enhanced presentation of empirical results. Furthermore, the PI has started working on Bayesian nested analysis that significantly improves the current practice of nested analysis.