Periodic Reporting for period 1 - PassiveValueMarkers (Identification and change of passive value markers in the laboratory and naturalistic environments)
Période du rapport: 2023-07-01 au 2025-12-31
Context and Overall Objectives
The project explores how humans assign value to individual items and options in the absence of explicit reinforcement or decision-making. Traditional decision-making research typically focuses on active choices and reward-based learning. However, we claim that much of our real-world preference formation happens passively, when we view or interact with stimuli without being asked to make decisions. The project investigates how these passive processes shape value representations in the brain and behavior.
To tackle this question, we use a multidisciplinary approach incorporating eye-tracking, functional MRI (fMRI), behavioral testing, and immersive virtual reality (VR) environments. The project aims to develop passive markers of value by examining gaze patterns, neural signatures, and behavior in highly controlled and naturalistic contexts. The insights generated can pave the way for innovations in marketing, mental health interventions, and human-computer interaction, among others.
Three major aims guide the work:
• Aim 1: Identify behavioral and neural passive markers of value using computational modeling.
• Aim 2: Analyze passive value markers in gaze path data using eye-tracking and fMRI.
• Aim 3: Test passive value construction in naturalistic virtual environments, including motion tracking.
Through this multifaceted approach, we aim to lay the groundwork for understanding how preferences are formed and can be manipulated, even without overt decision-making.
The project explores how humans assign value to individual items and options in the absence of explicit reinforcement or decision-making. Traditional decision-making research typically focuses on active choices and reward-based learning. However, we claim that much of our real-world preference formation happens passively, when we view or interact with stimuli without being asked to make decisions. The project investigates how these passive processes shape value representations in the brain and behavior.
To tackle this question, we use a multidisciplinary approach incorporating eye-tracking, functional MRI (fMRI), behavioral testing, and immersive virtual reality (VR) environments. The project aims to develop passive markers of value by examining gaze patterns, neural signatures, and behavior in highly controlled and naturalistic contexts. The insights generated can pave the way for innovations in marketing, mental health interventions, and human-computer interaction, among others.
Three major aims guide the work:
• Aim 1: Identify behavioral and neural passive markers of value using computational modeling.
• Aim 2: Analyze passive value markers in gaze path data using eye-tracking and fMRI.
• Aim 3: Test passive value construction in naturalistic virtual environments, including motion tracking.
Through this multifaceted approach, we aim to lay the groundwork for understanding how preferences are formed and can be manipulated, even without overt decision-making.
The project is on schedule and has made substantial progress in all three aims:
Aim 1: Passive Value via Computational Modelling
• Published a top 10% paper (Salomon et al., 2025, JEP:G) identifying a behavioral computational marker of value change at the item level.
• Collected behavioral data (n=40) comparing the impact of various cue types (auditory/visual, attention/natural) on preference change.
• Reanalyzed prior fMRI datasets (n=138) to link neural activations to a Bayesian learning parameter (Theta slope), identifying correlates with length of training in the frontal gyrus, occipital cortex, and caudate, among others.
Aim 2: Eye-Tracking-Based Passive Markers
• Recorded eye-tracking data from 75 participants viewing 150 individual images of faces and snacks, followed by subjective preference ratings.
• Applied deep learning models (RNNs, CNNs) to gaze data, finding that temporal dynamics in eye movements can successfully predict preferences, highlighting the promise of gaze as a passive marker of value.
Aim 3: Passive Value in Naturalistic Virtual Environments
• Developed multiple VR-based tasks using Unity on the Meta Quest Pro headset to collect data on gaze, body position, and hand kinematics.
• Replicated a prior study identifying passive value markers in gaze during VR exposure.
• Created and deployed a museum version in VR paradigm and a corresponding real web based version to validate lab findings in naturalistic settings.
• Initiated a series of VR studies, including a spatial maze task (n=32), a free-roaming museum tour (n=60), novel multisensory valuation tasks, augmented reality preferences and single item valuation in VR.
Infrastructure and Team
• Established a sophisticated QA pipeline for VR data (face tracking, motion, gaze).
• Built cross-disciplinary collaborations with experts in neurobiology, computer science, and psychology.
• The team comprises data scientists, designers, and research assistants, many trained in biology, psychology, and computational and data sciences.
Aim 1: Passive Value via Computational Modelling
• Published a top 10% paper (Salomon et al., 2025, JEP:G) identifying a behavioral computational marker of value change at the item level.
• Collected behavioral data (n=40) comparing the impact of various cue types (auditory/visual, attention/natural) on preference change.
• Reanalyzed prior fMRI datasets (n=138) to link neural activations to a Bayesian learning parameter (Theta slope), identifying correlates with length of training in the frontal gyrus, occipital cortex, and caudate, among others.
Aim 2: Eye-Tracking-Based Passive Markers
• Recorded eye-tracking data from 75 participants viewing 150 individual images of faces and snacks, followed by subjective preference ratings.
• Applied deep learning models (RNNs, CNNs) to gaze data, finding that temporal dynamics in eye movements can successfully predict preferences, highlighting the promise of gaze as a passive marker of value.
Aim 3: Passive Value in Naturalistic Virtual Environments
• Developed multiple VR-based tasks using Unity on the Meta Quest Pro headset to collect data on gaze, body position, and hand kinematics.
• Replicated a prior study identifying passive value markers in gaze during VR exposure.
• Created and deployed a museum version in VR paradigm and a corresponding real web based version to validate lab findings in naturalistic settings.
• Initiated a series of VR studies, including a spatial maze task (n=32), a free-roaming museum tour (n=60), novel multisensory valuation tasks, augmented reality preferences and single item valuation in VR.
Infrastructure and Team
• Established a sophisticated QA pipeline for VR data (face tracking, motion, gaze).
• Built cross-disciplinary collaborations with experts in neurobiology, computer science, and psychology.
• The team comprises data scientists, designers, and research assistants, many trained in biology, psychology, and computational and data sciences.
Results Beyond the State of the Art
This project has produced multiple innovations:
• Computational Marker of Passive Preference Change: A published Bayesian model demonstrates that individual value changes can be tracked computationally in the absence of reinforcement, an important step at the individual items level.
• VR-Based Preference Research: Developing a first-of-its-kind QA and analysis pipeline for rich, high-dimensional VR behavioral data. The lab is among the first globally to use passive multi-sensory VR environments to track value assignment.
• Eye Tracking with Deep Learning: Demonstrated initial evidence that recurrent neural networks can reliably decode preferences from passive viewing data of single images. Upon proper replication and elaboration this can lead to advances both methodological and potential applications in e.g. mental health.
• Data Sharing and Open Science: Task codes and datasets are being shared openly to promote transparency and reuse. The lab leads in implementing reproducible, high-quality data standards for VR-based psychological research.
These results provide a springboard for further research in cognitive neuroscience, and human-computer interaction.
This project has produced multiple innovations:
• Computational Marker of Passive Preference Change: A published Bayesian model demonstrates that individual value changes can be tracked computationally in the absence of reinforcement, an important step at the individual items level.
• VR-Based Preference Research: Developing a first-of-its-kind QA and analysis pipeline for rich, high-dimensional VR behavioral data. The lab is among the first globally to use passive multi-sensory VR environments to track value assignment.
• Eye Tracking with Deep Learning: Demonstrated initial evidence that recurrent neural networks can reliably decode preferences from passive viewing data of single images. Upon proper replication and elaboration this can lead to advances both methodological and potential applications in e.g. mental health.
• Data Sharing and Open Science: Task codes and datasets are being shared openly to promote transparency and reuse. The lab leads in implementing reproducible, high-quality data standards for VR-based psychological research.
These results provide a springboard for further research in cognitive neuroscience, and human-computer interaction.