Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence

Projektinformationen

CrossEDGE

ID Finanzhilfevereinbarung: 101066503

DOI

10.3030/101066503

Das Projekt endete am 31 Mai 2025

EK-Unterschriftsdatum 11 Juli 2022

Startdatum 1 August 2022

Enddatum 31 August 2026

Finanziert unter

• Marie Skłodowska-Curie Actions (MSCA)

Gesamtkosten

Keine Daten

EU-Beitrag

€ 203 464,32

Koordiniert durch

UNIVERSITEIT UTRECHT
Netherlands

Periodic Reporting for period 1 - CrossEDGE (Crossmodal estimation of decisions, guesses and delays in decision making: a collaboration between psychology, physiology and artificial intelligence)

Berichtszeitraum: 2022-08-01 bis 2026-08-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

Decision‑making under time pressure is a fundamental cognitive function that shapes everyday behaviour, clinical diagnosis, education, and the design of autonomous systems. Yet, conventional behavioural analyses collapse thousands of milliseconds of neural activity into a single reaction‑time measure, obscuring the rapid succession of mental operations that actually drive a choice. The central goal of this was to combine trial-by-trial electroencephalographic (EEG) recordings with behavioral data and translate this into a computational framework capable of labeling each decision as a guess, an evidence-accumulation process, or a delayed response.

The project pursued three tightly coupled objectives:
1 - Derive a model integrating physiological and behavioral data for the characterization of cognitive events in the EEG at the trial level
2 - Use the detected EEG events to understand the cognitive processes in decision-making
3 - Combine the outputs of the first two objectives into a generative framework that predicts the proportion of each strategy used across different speed‑accuracy regimes, thereby offering a quantitative bridge between neural dynamics and behavioural policy

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

In the first phase, we developed the Hidden Multivariate Pattern (HMP) model—a method that reliably detects cognitive events on a single-trial basis. The framework combines multivariate sensor patterns with probabilistic timing models to infer latent cognitive events. HMP was validated on both simulated and open datasets. The results were published in a peer-reviewed article (Trial-by-trial detection of cognitive events in neural time-series, Weindel, van Maanen & Borst, 2025).

The second work package focused on decomposing decision-making into its constituent cognitive processes. Two experiments were conducted with around thirty participants each. The first study was published in eLife as a reviewed preprint (Decision-making components and times revealed by the single-trial electro-encephalogram, Weindel, Borst & van Maanen). This study showed that we can decompose the reaction time in a decision-making task at the trial level into different periods of decision processing. These periods were then used to fit evidence accumulation models, improving parameter estimation beyond behavioral data alone. The second dataset, in preparation, builds on this first study to extend leading decision models, the evidence accumulation models.

The third work package aimed to integrate decision models beyond evidence accumulation models, e.g. guessing, evidence accumulation, and delayed decision,. However, the EEG data did not yield sufficiently distinct neural signatures for these strategies. As a result, the project pivoted to focus on strengthening and generalizing the HMP framework. This pivot preserved the core scientific contribution and led to a tool already being adopted by several European research groups.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

The project showed that the Hidden Multivariate Pattern (HMP) model markedly improves the interpretability of EEG data. In the second objective, HMP not only quantified the number of cognitive events underlying each decision but also pinpointed their timing on a trial‑by‑trial basis. By extracting reliable single‑event signatures despite low signal‑to‑noise ratios, the HMP model appears as a powerful for cognitive‑neuroscience research. Although the current work centered on decision‑making, future studies should explore its applicability to other domains—such as language processing, motor control, and clinical diagnostics—to fully evaluate its broader impact across the neuroscientific landscape.

Representation of the Hidden Multivariate Pattern model