Periodic Reporting for period 4 - UNIFY (A Unified Framework for the Assessment and Application of Cognitive Models)
Reporting period: 2022-07-01 to 2022-12-31
Cognitive models formalize substantive theory about how people reason, learn, decide, and act.
Cognitive models also serve as measurement tools that explain observed behavior in terms of constituent
psychological processes. Because of their unique ability to estimate latent processes, cognitive models are
increasingly applied throughout cognitive neuroscience and clinical psychology. Despite their theoretical
appeal and growing popularity, however, the field of cognitive modeling presents an often bewildering
proliferation of ideas and techniques. Current applications appear idiosyncratic, and the state-of-the-art
remains unclear. This lack of systematicity makes it difficult for researchers and practitioners to develop,
understand, and apply important cognitive models.
The main goal of the Advanced ERC project “UNIFY” was to provide a unified, systematic treatment of cognitive
models. By adhering to the basic principles of Bayesian inference we developed new methods and
proposed new procedures to address core modeling questions. The innovation took place both on an
abstract level (through the activities of a Quantitative Development Team) and on a concrete, model-specific
level (through the activities of a Core Applications Team).
In the UNIFY project we set new standards for cognitive modeling. By advancing a more systematic treatment of
uncertainty we aimed to push cognitive model evaluation and application to the next level. A secondary goal was to
increase the availability and boost the impact of the project by making the new procedures available in the free
software packages R and JASP. Primary new technology was developed to test models and quantify the
associated uncertainty. The project also revealed that experts often disagree on the preferred modeling approach,
underscoring the need to apply multiple models or multiple analysis teams.
Cognitive models also serve as measurement tools that explain observed behavior in terms of constituent
psychological processes. Because of their unique ability to estimate latent processes, cognitive models are
increasingly applied throughout cognitive neuroscience and clinical psychology. Despite their theoretical
appeal and growing popularity, however, the field of cognitive modeling presents an often bewildering
proliferation of ideas and techniques. Current applications appear idiosyncratic, and the state-of-the-art
remains unclear. This lack of systematicity makes it difficult for researchers and practitioners to develop,
understand, and apply important cognitive models.
The main goal of the Advanced ERC project “UNIFY” was to provide a unified, systematic treatment of cognitive
models. By adhering to the basic principles of Bayesian inference we developed new methods and
proposed new procedures to address core modeling questions. The innovation took place both on an
abstract level (through the activities of a Quantitative Development Team) and on a concrete, model-specific
level (through the activities of a Core Applications Team).
In the UNIFY project we set new standards for cognitive modeling. By advancing a more systematic treatment of
uncertainty we aimed to push cognitive model evaluation and application to the next level. A secondary goal was to
increase the availability and boost the impact of the project by making the new procedures available in the free
software packages R and JASP. Primary new technology was developed to test models and quantify the
associated uncertainty. The project also revealed that experts often disagree on the preferred modeling approach,
underscoring the need to apply multiple models or multiple analysis teams.
The primary achievement concerns the development and application of two underused but highly promising statistical techniques – bridge sampling and model-averaging. With bridge sampling, researchers can compute a model’s predictive performance in an efficient and reliable manner. With model-averaging, researchers can base their overall conclusion on many models simultaneously: each model’s contribution is combined with that of the others, with its influence weighted with past predictive performance. In addition, considerable progress has been made to make JASP suitable as a general-purpose software program for cognitive models. Specifically, much work has been done on making it easy to add modules, on obtaining the underlying R code, and on developing a module that allows probabilistic programming with the help of a graphical user interface. We have developed a state-of-the-art routine for the estimation of the drift-diffusion model, and we have extended models for the time it takes to stop by striking a balance between psychological interpretability and psychometric tractability. Considerable effort was expended to write a course book on Bayesian inference. The activities from UNIFY have led to an in-depth discussion on cognitive modeling that revealed fundamental differences of opinion.
Other than the progress outlined above, the UNIFY project also resulted in new insights on design analyses (which Bayesians may conduct on-the-fly, as new data accumulate) and on the relation between frequentist p-values and the Bayes factor.
In general, the UNIFY project attempted to make systematic progress by taking Bayes' theorem as a point of departure, and address relevant scientific modeling questions within that framework.
In general, the UNIFY project attempted to make systematic progress by taking Bayes' theorem as a point of departure, and address relevant scientific modeling questions within that framework.