Periodic Reporting for period 1 - PROCESSHETEROGENEITY (Understanding the Processes Underlying Societal Threats using Novel Cluster-based Methods)
Reporting period: 2022-10-01 to 2025-03-31
Social scientists are eager to answer questions about relations between constructs like beliefs or values. For example, do values affect climate change beliefs? Do perceived threats predict political beliefs? It is important to study what drives beliefs about climate, politics, and other important social constructs. Large-scale survey data is gathered to do so.
Using regression to answer the questions ignores that constructs are not directly observable, but measured by survey items containing measurement error (challenge 1). Not correcting for this causes the studied effects to be underestimated and conclusions to be misguided.
When many groups are involved – such as many countries in the European Social Survey – the underlying processes likely differ across groups. For example, drivers of climate change beliefs may differ for countries experiencing extreme weather. Group-specific or multilevel analyses result in numerous group-specific regression slopes or random effects, making it hard to find which regression effects are different or similar for which groups (challenge 2).
Across many groups, the constructs’ measurement is often inequivalent or ‘non-invariant’, for example, due to translation (challenge 3). A measurement model indicates how items measure a construct and disregarding non-invariance in this model invalidates the comparison of effects among constructs (i.e. one may find differences that are actually due to non-invariance).
By tackling challenges 1-3, the proposed mixture multigroup structural equation modelling framework provides the tools to break new ground in understanding what drives constructs like polarized beliefs. A clustering finds subsets of groups with common processes. Flexible measurement models account for non-invariance so that the clustering focuses on the processes and is unaffected by differences in the measurement model. The methods will be implemented in freely available software.
Using regression to answer the questions ignores that constructs are not directly observable, but measured by survey items containing measurement error (challenge 1). Not correcting for this causes the studied effects to be underestimated and conclusions to be misguided.
When many groups are involved – such as many countries in the European Social Survey – the underlying processes likely differ across groups. For example, drivers of climate change beliefs may differ for countries experiencing extreme weather. Group-specific or multilevel analyses result in numerous group-specific regression slopes or random effects, making it hard to find which regression effects are different or similar for which groups (challenge 2).
Across many groups, the constructs’ measurement is often inequivalent or ‘non-invariant’, for example, due to translation (challenge 3). A measurement model indicates how items measure a construct and disregarding non-invariance in this model invalidates the comparison of effects among constructs (i.e. one may find differences that are actually due to non-invariance).
By tackling challenges 1-3, the proposed mixture multigroup structural equation modelling framework provides the tools to break new ground in understanding what drives constructs like polarized beliefs. A clustering finds subsets of groups with common processes. Flexible measurement models account for non-invariance so that the clustering focuses on the processes and is unaffected by differences in the measurement model. The methods will be implemented in freely available software.
From the mixture multigroup SEM framework, the novel method Mixture Multilevel SEM (MixML-SEM) was developed, where random effects capture differences in measurement parameters across the groups, so that they don’t affect the clustering. In an extensive simulation study, it was compared to multilevel SEM (ML-SEM), which is, for many researchers, the preferred method for comparing structural relations (i.e. regression relations among latent variables or latent processes) across many groups. We showed that ML-SEM produces group-specific structural relations that are incorrectly estimated for smaller groups, and cumbersome to compare. In contrast, MixML-SEM provides accurate estimates of the structural relations of interest and an efficient comparison of groups in terms of structural relations by assigning groups with the same structural relations to the same cluster. In this way, one only needs to compare the relations between the clusters of groups (rather than between individual groups). Finding a clustering of groups specifically focused on the structural relations, and unaffected by differences in measurement, was a key objective of the project, which is thus achieved.
A second method of the framework, Mixture Multigroup Bayesian SEM (MixMG-BSEM), was also developed, where small differences in measurement parameters are captured by small-variance priors around the measurement parameters. A good performance was found in a large simulation study, which also showed that MixMG-BSEM is quite robust to the choice of the prior variances.
In order to accommodate mixture multigroup SEM methods with an exploratory measurement model (where it is unknown beforehand which items are measuring which latent variables), we are evaluating how structural relations should be compared across groups when the measurement model is exploratory.
Users have to specify the number of clusters for the data set at hand (model selection). In an extensive simulation study, we confirmed that existing techniques for selecting the number of clusters succeed in correctly identifying the number of clusters for MixML-SEM. These results generalize to other methods of the framework.
A second method of the framework, Mixture Multigroup Bayesian SEM (MixMG-BSEM), was also developed, where small differences in measurement parameters are captured by small-variance priors around the measurement parameters. A good performance was found in a large simulation study, which also showed that MixMG-BSEM is quite robust to the choice of the prior variances.
In order to accommodate mixture multigroup SEM methods with an exploratory measurement model (where it is unknown beforehand which items are measuring which latent variables), we are evaluating how structural relations should be compared across groups when the measurement model is exploratory.
Users have to specify the number of clusters for the data set at hand (model selection). In an extensive simulation study, we confirmed that existing techniques for selecting the number of clusters succeed in correctly identifying the number of clusters for MixML-SEM. These results generalize to other methods of the framework.
The most significant achievement of this project so far, is the stepwise estimation of the novel methods, where the measurement model is estimated first (dealing with differences) and the structural model is estimated in a next step (clustering groups on their structural relations). The stepwise estimation has worked out much better than anticipated in advance, to the degree that we no longer need (to compare with) a one-step estimation. The stepwise estimation streamlines computation time and maximizes flexibility. The latter to the extent that different estimators can be used in the measurement model and structural model steps of the methods’ estimation, even using different software. Some things would not even be possible with the one-step estimation, like an extension for dealing with ordinal items, which makes this an achievement that is beyond the state of the art. Since the clustering of the groups requires maximum likelihood estimation, a one-step estimation would also imply maximum likelihood estimation for the measurement model with ordinal items, which is known to be very time consuming. So adapting the stepwise estimation methods for SEM to the multigroup case and making them work for the MixMG-SEM methods is a very important achievement of this project.