Learning heuristics in preference elicitation tasks: insights from behavioural, computational and neurobiological investigations

Project Information

PREF LEARNHEUR

Grant agreement ID: 657904

DOI

10.3030/657904

Project terminated on 30 January 2016

EC signature date 7 April 2015

Start date 1 May 2015

End date 30 April 2017

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 165 598,80

EU contribution

€ 165 598,80

Coordinated by

UNIVERSITEIT VAN AMSTERDAM
Netherlands

Periodic Reporting for period 1 - PREF LEARNHEUR (Learning heuristics in preference elicitation tasks: insights from behavioural, computational and neurobiological investigations)

Reporting period: 2015-05-01 to 2017-04-30

Summary of the context and overall objectives of the project

Decision theory assumes that in order to make a choice, individuals attribute values to available options, compare those values and select the option with the highest. In behavioral economics, the values attributed to options presumably integrate individuals’ attitude (or preference) toward risk, ambiguity, or delay. Hence, in order to estimate, e.g. individuals’ attitude toward risk, the classical strategy is to offer succession of binary choices between risky lotteries, and estimate how people “value” gains and probabilities. However, the succession of choices faced by individuals during those classical “preference elicitation” tasks might trigger the emergence of additional strategies and heuristics, implemented to perform those tasks in a fast, yet adaptive manner. Then, instead of computing complex values for available options, individual might learn to apply simple rules, such as: “focus on the probabilities –i.e. ignore information about potential gains- and choose the safe option.” This project aimed at investigating the development of such heuristic, from a behavioral, computational and neural perspective.

Estimating individuals’ preferences has become crucial in many different fields: in social sciences, where it constitutes the micro-economic basis of macro-economic models and policies; in clinical sciences, where preferences are increasingly used as diagnostic tools, e.g. in neuro-psychiatry; in neurosciences, where preferences are used as an important variable to investigate the neural computations at stake in decision-making. Understanding if and when people change strategies during preference elicitation tasks –e.g. by implementing heuristic rules rather than following the canonical “value-based” strategy–can help to refine preference elicitation, hence can have a significant impact in society.

The objective are
1) Capturing, behaviorally, the emergence of heuristics in a widely-used instantiation of binary economic choices task,
2) Modelling this emergence, using a computational model
3) Exploring its neural implementation in humans.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

According to the initial plan, the work performed during this shortened MSCA fellowship (8 months) mostly investigated questions related to the first objective (WP1). This resulted in several important developments.

1. The idea that two competing strategies (value-based and heuristics) might contribute to the final behavior has led the applicant to investigate the notion of Mixtures in Value-Based Decision-Making. This has taken the form of a collaboration with Dr. L van Maanen at the host institution, who develops methodologies to assess and test the presence of such mixtures. While adapting those methodologies to preference-elicitation tasks, important developments have been undertaken, which resulted in a methodological paper, published in PLoS one:
REF: van Maanen, L., Couto, J., & Lebreton, M. (2016). Three Boundary Conditions for Computing the Fixed-Point Property in Binary Mixture Data. PloS one, 11(11), e0167377.

2. A modified version of the main experimental test (initial proposal Part B - Section 2.1 - WP1) has been designed, and piloted. Data has been collected, the results are currently being analyzed, and a manuscript is under preparation. Preliminary results seem to reveal that individuals indeed implement heuristics, by increasing the decision-weight on their default-option. The manuscript will be uploaded on the preprint server when written, and will be submitted to an international peer-reviewed journal.
REF: Couto, J., van Maanen, L., Lebreton, M. (in prep). Investigating utility vs heuristics processes in value-based decision-making.

Ancillary projects: In addition, the applicant has contributed to the following project
1. The idea that people may use simple rules is not restricted to preference elicitation tasks, and can be applied to other decision situations, like in reinforcement –learning tasks. In a collaboration with Dr. S. Plaminteri, the applicant contributed to show that some individuals focus on positive reinforcements and tend to ignore negative reinforcement, generating an optimism bias. T
REF: Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S. & Palminteri, S. Behavioural and neural characterization of optimistic reinforcement learning. Nature Human Behaviour. 1, 0067 (2017).

2. As mentioned in the initial proposal (Part B - Section 4.2) the applicant has leveraged the current framework to investigate the topic of “neuro-computational heterogeneity in value-based behavior”, i.e. brain-behavior individual difference. A first output, written as an important conceptual and opinion piece, has been uploaded on biorXiv, and is currently under review at international peer-reviewed journal.
Lebreton, M., & Palminteri, S. (2016). When are inter-individual brain-behavior correlations informative? bioRxiv, 036772.

All those results have been communicated using various media: published articles in international peer-reviewed journals, draft manuscripts uploaded on open access preprint servers (biorXiv), posters at conferences (Annual Meeting of the Society for Neuroeconomics 09/2015 Miami; Annual Meeting of the Organization for Human Brain Mapping 06/2015; Symposium on the Biology of Decision-Making 05/2015), invited talks at symposium (e.g. 2016 - Amsterdam Brain & Cognition Symposium), and invited talks at different labs.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The core hypothesis of this project propose that choices collected during preference elicitation tasks result from a dynamical competition between value-based and heuristic processes.
Important methodological and experimental progress has been made, regarding this hypothesis.
• A methodological study completed during this project (van Maanen, et al. (2016) PLoS one) refines recent analytical tools to assess the presence of such mixtures of processes in the behavior. Other methodological follow-ups are currently in progress. This work will have broader, fundamental implications, given that large fractions of experimental studies in psychology, economics and neuroscience are rooted in the Dual-process theories (i.e. assume that our behavior is driven by a mixture of two competing processes), hence will need analytical tool to statistically assess those assumptions.
• Besides those methodological developments, the first pieces of evidence, suggesting that people gradually adapt their computations and learn to use heuristic rules, have been collected. These studies concern preference elicitation task (Couto, et al. in prep), but also reinforcement learning (Lefebvre, et al. (2017) Nature Human Behaviour). This conclusions will have fundamental implication for all the experimental fields using model-based assessment of individual characteristics and/or preferences, such as social sciences, clinical sciences, and neuro-economics.

Besides, a critical evaluation of analytical strategies regarding inter-individual differences in brain-behavior has been drafted. This manuscript is expected to have a large impact in the neuroimaging community (in fundamental cognitive neurosciences, but also in clinical sciences), and already attracted a lot of attention online, since it has been uploaded on the biorXiv preprint server. (Lebreton and Palminteri (2016) biorXiv).

Figure 1 for Van Maanen, Couto and Lebreton (2013)