Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence

Información del proyecto

CrossEDGE

Identificador del acuerdo de subvención: 101066503

DOI

10.3030/101066503

Proyecto finalizado el 31 Mayo 2025

Fecha de la firma de la CE 11 Julio 2022

Fecha de inicio 1 Agosto 2022

Fecha de finalización 31 Agosto 2026

Financiado con arreglo a

• Marie Skłodowska-Curie Actions (MSCA)

Coste total

Sin datos

Aportación de la UE

€ 203 464,32

Coordinado por

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - CrossEDGE (Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence)

Período documentado: 2022-08-01 hasta 2026-08-31

Resumen del contexto y de los objetivos generales del proyecto

Decision‑making under time pressure is a fundamental cognitive function that shapes everyday behaviour, clinical diagnosis, education, and the design of autonomous systems. Yet, conventional behavioural analyses collapse thousands of milliseconds of neural activity into a single reaction‑time measure, obscuring the rapid succession of mental operations that actually drive a choice. The central goal of this was to combine trial-by-trial electroencephalographic (EEG) recordings with behavioral data and translate this into a computational framework capable of labeling each decision as a guess, an evidence-accumulation process, or a delayed response.

The project pursued three tightly coupled objectives:
1 - Derive a model integrating physiological and behavioral data for the characterization of cognitive events in the EEG at the trial level
2 - Use the detected EEG events to understand the cognitive processes in decision-making
3 - Combine the outputs of the first two objectives into a generative framework that predicts the proportion of each strategy used across different speed‑accuracy regimes, thereby offering a quantitative bridge between neural dynamics and behavioural policy

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

In the first phase, we developed the Hidden Multivariate Pattern (HMP) model—a method that reliably detects cognitive events on a single-trial basis. The framework combines multivariate sensor patterns with probabilistic timing models to infer latent cognitive events. HMP was validated on both simulated and open datasets. The results were published in a peer-reviewed article (Trial-by-trial detection of cognitive events in neural time-series, Weindel, van Maanen & Borst, 2025).

The second work package focused on decomposing decision-making into its constituent cognitive processes. Two experiments were conducted with around thirty participants each. The first study was published in eLife as a reviewed preprint (Decision-making components and times revealed by the single-trial electro-encephalogram, Weindel, Borst & van Maanen). This study showed that we can decompose the reaction time in a decision-making task at the trial level into different periods of decision processing. These periods were then used to fit evidence accumulation models, improving parameter estimation beyond behavioral data alone. The second dataset, in preparation, builds on this first study to extend leading decision models, the evidence accumulation models.

The third work package aimed to integrate decision models beyond evidence accumulation models, e.g. guessing, evidence accumulation, and delayed decision,. However, the EEG data did not yield sufficiently distinct neural signatures for these strategies. As a result, the project pivoted to focus on strengthening and generalizing the HMP framework. This pivot preserved the core scientific contribution and led to a tool already being adopted by several European research groups.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

The project showed that the Hidden Multivariate Pattern (HMP) model markedly improves the interpretability of EEG data. In the second objective, HMP not only quantified the number of cognitive events underlying each decision but also pinpointed their timing on a trial‑by‑trial basis. By extracting reliable single‑event signatures despite low signal‑to‑noise ratios, the HMP model appears as a powerful for cognitive‑neuroscience research. Although the current work centered on decision‑making, future studies should explore its applicability to other domains—such as language processing, motor control, and clinical diagnostics—to fully evaluate its broader impact across the neuroscientific landscape.

Representation of the Hidden Multivariate Pattern model