Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence

Informations projet

CrossEDGE

N° de convention de subvention: 101066503

DOI

10.3030/101066503

Projet clôturé le 31 Mai 2025

Date de signature de la CE 11 Juillet 2022

Date de début 1 Août 2022

Date de fin 31 Août 2026

Financé au titre de

• Marie Skłodowska-Curie Actions (MSCA)

Coût total

Aucune donnée

Contribution de l’UE

€ 203 464,32

Coordonné par

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - CrossEDGE (Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence)

Période du rapport: 2022-08-01 au 2026-08-31

Résumé du contexte et des objectifs généraux du projet

Decision‑making under time pressure is a fundamental cognitive function that shapes everyday behaviour, clinical diagnosis, education, and the design of autonomous systems. Yet, conventional behavioural analyses collapse thousands of milliseconds of neural activity into a single reaction‑time measure, obscuring the rapid succession of mental operations that actually drive a choice. The central goal of this was to combine trial-by-trial electroencephalographic (EEG) recordings with behavioral data and translate this into a computational framework capable of labeling each decision as a guess, an evidence-accumulation process, or a delayed response.

The project pursued three tightly coupled objectives:
1 - Derive a model integrating physiological and behavioral data for the characterization of cognitive events in the EEG at the trial level
2 - Use the detected EEG events to understand the cognitive processes in decision-making
3 - Combine the outputs of the first two objectives into a generative framework that predicts the proportion of each strategy used across different speed‑accuracy regimes, thereby offering a quantitative bridge between neural dynamics and behavioural policy

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

In the first phase, we developed the Hidden Multivariate Pattern (HMP) model—a method that reliably detects cognitive events on a single-trial basis. The framework combines multivariate sensor patterns with probabilistic timing models to infer latent cognitive events. HMP was validated on both simulated and open datasets. The results were published in a peer-reviewed article (Trial-by-trial detection of cognitive events in neural time-series, Weindel, van Maanen & Borst, 2025).

The second work package focused on decomposing decision-making into its constituent cognitive processes. Two experiments were conducted with around thirty participants each. The first study was published in eLife as a reviewed preprint (Decision-making components and times revealed by the single-trial electro-encephalogram, Weindel, Borst & van Maanen). This study showed that we can decompose the reaction time in a decision-making task at the trial level into different periods of decision processing. These periods were then used to fit evidence accumulation models, improving parameter estimation beyond behavioral data alone. The second dataset, in preparation, builds on this first study to extend leading decision models, the evidence accumulation models.

The third work package aimed to integrate decision models beyond evidence accumulation models, e.g. guessing, evidence accumulation, and delayed decision,. However, the EEG data did not yield sufficiently distinct neural signatures for these strategies. As a result, the project pivoted to focus on strengthening and generalizing the HMP framework. This pivot preserved the core scientific contribution and led to a tool already being adopted by several European research groups.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The project showed that the Hidden Multivariate Pattern (HMP) model markedly improves the interpretability of EEG data. In the second objective, HMP not only quantified the number of cognitive events underlying each decision but also pinpointed their timing on a trial‑by‑trial basis. By extracting reliable single‑event signatures despite low signal‑to‑noise ratios, the HMP model appears as a powerful for cognitive‑neuroscience research. Although the current work centered on decision‑making, future studies should explore its applicability to other domains—such as language processing, motor control, and clinical diagnostics—to fully evaluate its broader impact across the neuroscientific landscape.

Representation of the Hidden Multivariate Pattern model