Range-adapting reinforcement learning and memory

Projektinformationen

RaReMem

ID Finanzhilfevereinbarung: 101043804

DOI

10.3030/101043804

EK-Unterschriftsdatum 22 September 2022

Startdatum 1 Januar 2023

Enddatum 31 Dezember 2027

Finanziert unter

• European Research Council (ERC)

Gesamtkosten

€ 1 995 826,00

EU-Beitrag

€ 1 995 826,00

Koordiniert durch

INSTITUT NATIONAL DE LA SANTE ET DE LA RECHERCHE MEDICALE
France

Periodic Reporting for period 1 - RaReMem (Range-adapting reinforcement learning and memory)

Berichtszeitraum: 2023-01-01 bis 2025-06-30

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

A growing body of research in perceptual and economic decision-making has shown that the subjective value of an option is not computed in isolation, but depends critically on the range of available alternatives—a phenomenon known as contextual valuation. However, despite the ecological relevance of this effect, most existing work has focused on artificially static and explicit contexts, leaving open the question of how value representations adapt in more naturalistic environments shaped by past experiences. This project aims to fill that gap by investigating whether, and how, outcome values learned from experience are dynamically rescaled through a process of range adaptation. Our central hypothesis is that such range adaptation is not a marginal feature, but a pervasive computational strategy that optimizes information processing in learning systems. By adapting the internal value scale to the distribution of experienced outcomes, the brain reduces computational costs and enhances local efficiency. However, this comes at a cost: the resulting value representations may become distorted, impairing the ability to generalize across novel contexts. To address this trade-off, we are developing and empirically validating a comprehensive theoretical and experimental framework. Through the integration of behavioral studies, computational modeling, and cross-cultural experiments, the project seeks to provide a rigorous account of range adaptation in human learning. Ultimately, our goal is to uncover fundamental computational constraints on human cognition and contribute to a deeper understanding of the mechanisms underlying bounded rationality. Beyond its implications for cognitive science, this work may inform broader discussions on decision-making, memory, and adaptability in complex environments.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

Since the start of the project, we have successfully launched all major research activities across the different work packages, advancing both the experimental and computational components in parallel. Our work has already yielded key scientific contributions that provide strong empirical support for the central hypotheses of the project. One major achievement is the development of a new reinforcement learning task that allowed us to disentangle competing theories of value normalization. Contrary to the widely held belief that human value representations rely on divisive normalization—an idea rooted in perceptual decision-making—we found compelling evidence in favor of range normalization. This result, published in eLife, offers a more accurate and ecologically valid account of context-dependent learning. In another study, published in Nature Human Behaviour, we investigated whether context sensitivity in value-based learning is a universal feature of human cognition or subject to cultural variation. By collecting data from participants in eleven countries with diverse socioeconomic backgrounds, we found that range adaptation is strikingly consistent across populations, in contrast to other cognitive processes such as risk aversion, which varied significantly by culture. These findings not only reinforce the robustness of context-dependent reinforcement learning but also demonstrate the effectiveness of our methodological approach, which combines rigorous task design, behavioral analysis, and computational modeling. The work to date confirms the feasibility and scientific value of the project, with results already influencing ongoing debates in cognitive neuroscience and psychology.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

The project has already produced several outcomes that push the boundaries of existing research in learning and decision-making. Among the most significant advances is the development of new behavioral tasks specifically designed to capture subtle forms of value adaptation, which had previously eluded empirical investigation. These paradigms have allowed us to experimentally isolate the computational mechanisms we seek to understand, providing a solid empirical foundation for theoretical progress. In parallel, we have created original computational tools that enable more precise modeling of learning processes, enhancing our ability to test and refine hypotheses about context-sensitive valuation. Importantly, this methodological progress has been complemented by substantive findings. For the first time, we have demonstrated that the core mechanisms of reinforcement learning—and their associated biases—remain highly stable across different cultural environments. This finding highlights the deep-rooted nature of these cognitive processes and strengthens the case for their evolutionary and computational significance. In keeping with our commitment to open science, we are making our tools and datasets progressively available to the broader research community through a dedicated online repository. These resources are expected to facilitate further discovery, replication, and interdisciplinary collaboration. Taken together, our results mark a clear step forward in understanding the trade-offs that shape human learning and decision-making, and they lay the groundwork for future research into the generalizability and adaptability of cognitive computations.

1: countries. 2: behavioural task. 3: behavioural results. 4: model parameter.