Skip to main content

Neural circuit dynamics underlying expectation and their impact on the variability of perceptual choices

Periodic Reporting for period 3 - PRIORS (Neural circuit dynamics underlying expectation and their impact on the variability of perceptual choices)

Reporting period: 2019-09-01 to 2021-02-28

In this project we investigate the mechanisms underlying the perception of the environment. In particular we investigate the neural basis of how experience shapes our perception, or how expectations formed by our previous actions and their outcomes, are (1) built, (2) combined with sensory information and (3) processed to yield decisions. Expectations is an often overlooked topic in sensory systems neuroscience, in which the hierarchical circuitry starting from sensory receptors all the way up to the cortex has been finely characterized in various modalities (e.g. vision, audition, tactile, …). Current models of perceptual decision making assume that, during psychophysical tasks in which subjects need to discriminate ambiguous stimuli, a sensory representation formed in early cortical areas is integrated by associative areas in order to yield choices. The choice variability ubiquitous to all choices rising from stimuli at the perceptual threshold is thought to come from neuronal variability at the level of population of cortical sensory neurons. We hypothesize however, that a fraction of this variability might be caused by variability in the prior expectations formed by subjects based on previous decisions.

Understanding the neural mechanisms of decision making is a fundamental goal in neuroscience. Despite the advancement done in the last two decades there are still fundamental questions unresolved. How we weight evidence to take choices and learn from past experiences and adapt to a changing world is a central function of the brain which is affected in multiple psychiatric and neurological disorders such as schizophrenia, obsessive compulsive disorder, addictions, Parkinson disease, etc. Characterising the neural circuit mechanisms underlying the weighting of evidence and the learning from previous experiences will establish a conceptual framework under which we will be able to develop hypothesis for what goes wrong in each of these disorders and what kind of treatments may be effective.

The overall goal of this project is to characterize how we form perceptual expectations from our previous experiences, which are the brain areas involved in their computation and how these expectation are combined with sensory evidence in order to yield decisions. We will use both rodents and humans in a novel two-alternative forced choice task that promotes the development and exploitation of prior expectations. We will characterize choice dynamics using behavioral modeling and statistical learning techniques. We will combine these behavioral tasks with electrophysiological and manipulation of neuronal activity using optogenetics and pharmacology. Our findings will shed light onto expectations, an instrumental part of our perceptual experiences which may be altered in certain brain pathologies.
In the project we have covered the following activities along the lines described in the Description of Action (DoA).

Activity 1 : Development of a high-throughput Open source behavioral system for rodents.
In order to automatise and optimise the very time-consuming process of rodent training in cognitive tasks, we developed a high-throughput behavioral platform. As the platform is composed of many behavioral boxes, it was necessary to adjust the cost of the equipment and the software used to control each of these boxes, as well as to automatise the process in the way that the “bandwidth” of a single researcher can be maximized to supervise the training of many animals. The novel high-throughput platform for rodent training is composed of (1) PyBpod, an open software suited for the design and control of behavioural experiments, (2) custom made hardware aimed to (i) be fully compatible with Linux, (ii) minimize the cost and physical space of each box, (iii) maximize the temporal precision of sensors and actuators, the performed number of trials per session, and the reproducibility of the data.

Activity 2 : Rats behavioural characterization using a Reinforcement Learning statistical model of expectations [within Aim 1.1 of the DoM].
Using this high-throughput behavioral training set-up, we have performed many experiments using our nove perceptual discrimination two-alternative forced choice task (2AFC). Using these behavioral data we have finely characterized the across-trials dynamics of expectation build-up and its combination with auditory stimuli in order to yield choices. The results of this analysis and modeling of the behavior have been published in Hermoso-Mendizabal et al 2018.

Activity 3 : Permanent or reversible Inactivation of specific brain areas during the task [within Aim 1.2. Of the DoA]
We have performed numerous pharmacological and lesion studies in rats performing the 2AFC task. To explore the role of association cortices and in the striatum, we permanently or transiently abolished the activity of medial prefrontal cortex (mPFC), the medial posterior parietal cortex (PPC) and dorso-medial striatum (DMS). For the permanent-abolishing approach we performed bilateral neurotoxic lesions in the mPFC of 4 animals and tested the performance of the animals in the task after recovery. For the transiently-abolishing approach we bilateraly injected muscimol (a drug that inactivates the activity at the level of an entire neural circuit transiently) in mPFC, PPC or DMS in additional animals. The performance of the animals under the effect of this drug can be contrasted with the same animal’s performance injected with saline solution. We found no significant effects when inactivating mPFC or PPC. We did find however a significant reduction of the expectation choice bias when inactivating the DMS.
Activity 4 : Psychophysical Experiments to investigate the temporal integration of stimulus evidence [withing Aim 2.1. Of the DoA].
Before characterizing the integration of expectation and sensory stimulus, we have spent some time studying how subjects temporally integrate the sensory stream of information. The canonical models stimulus integration are based on what is called a diffusion process of a particle in a potential. The shape of this potential is what determines the integration dynamics of the system. We did psychophysical experiments in both humans and rats where we interleaved different magnitude of the stimulus fluctuations. This manipulation allowed us to test whether the perceptual decision relies more in the initial evidence of the stimulus (a Primacy effect) or in contrast is based on the late evidence (a Recency effect). The results of these experiments show that both humans (n=16) and rats (n=5) have very heterogeneous strategies to solve these tasks, ranging from Primacy to Recency. We also tested the impact of the stimulus fluctuation on the categorization accuracy as Double-Well attractor models predict a non-monotonically relation of the performance with the stimulus fluctuations. Again some humans participants and rats show this behaviour while other show a plateau follow by a decreases in performance. All these strategies might be explained by the double well potential in different regimes of integration and can not be explained by other traditional models that assume bounded or unbounded perfect integration. Due to the heterogeneity across subjects exhibited by the data, we are currently developing a new method that can fit the best potential function to each individual subject in order to find the mechanisms responsible of this heterogeneity.

Activity 5 : Modeling of the effect of expectations in the integration of the stimulus [within Aim 2.1 of the DoA]
To quantitatively describe the effect of expectations in the integration of the stimulus, we first studied rats’ reaction times (RTs), defined as the time they take to accumulate sufficient evidence to make the final decision during our 2AFC task. We found that RTs were determined by the combination of two processes: 1) an urgency signal that estimates the onset of the response window that coincides with the stimulus onset, and 2) the integration of the stimulus up to a bound following the standard dynamics of the canonical Drift Diffusion model (DDM). The first process, named “urgency integrator”, corresponds to both temporal signal that estimates the duration of the fixation window during which animals cannot respond. The urgency signal is captured by a single-constant-threshold Drift Diffusion Model (DDM). The second process, called “stimulus integrator”, is the standard accumulation of sensory evidence, modeled as a two-constant-bound DDM. In the context of this novel Dual DDM (i.e. D2M), rats’ RTs are set on a trial-by-trial basis by the first among the two integrators that reach a bound, whereas the choice is always determined by the sensory evidence accumulated in the stimulus integrator. In sum, this model allows to quantitatively characterize the combination of priors and stimulus integration

Activity 6 : Changes of mind in rats [[within Aim 2.1 of the DoA]
As a way to gain deeper understanding of the dynamics of integration of the prior expectation and stimulus we used DeepCut, a novel video-tracking algorithm that uses Deep Convolutional neural networks in order to track certain positions in the body of our animals. We tracked subjects choice trajectory and defined Changes of Mind (CoM) as those trials when the subjects final choice was opposed to its initial trajectory.
We found that, in some trials, rats take expressed responses (defined as those with RT < 100 ms) based on the prior expectation build from previous trials. Given this initial decision, depending on how strong was the prior and how strong is the evidence against the initial decision, rats can revert this initial trajectory and make a CoM. As expected, CoMs are more likely to happen when prior evidence is not very large and when the stimulus evidence is large and opposed. We have already recorded electrophysiological data from the Prefrontal cortex in one animal (premotor area M2) but the data remains to be analyzed. We will implant another three rats in the coming months.

Activity 7 : Electrophysiological experiments in the striatum [within Aim 1.3]
Given that inactivation of mPFC with muscimol or irreversible lesions had little effect on the animal’s performance during the task, we focus on the one area that we did see impairments when inactivated using muscimol: the dorso-medial striatum (DMS). We implanted five rats with tetrodes and recorded populations of neurons during the task during many sessions (up to six months of recordings). The analysis of the information encoded in these neurons reveals that the relevant variables to encode the transition evidence (the internal estimate of the animals to repeat or alternate their previous response) and the choice bias derived from it, are encoded in the DMS. We found one type of neurons encoded choice side from the response initiation until the next trial response. Importantly however, this encoding was only carried over to the next trial after rewarded responses. A second population of neurons encoded the transition evidence, maintaining this information after both correct and error responses. Finally, a third population encoded the Left-Right choice bias derived from the transition evidence. As this bias resulted from the combination of the transition evidence and the previous choice, it disappeared after errors, consistently with the animals behavior and the post-error vanishing of the neural representation of choices. Together, these results suggest that DMS dynamically encodes the relevant variables that are necessary to build and maintain expectations and use them to modulate behavior.
In addition to the recordings from DMS we have also recorded from the mPFC of three rats but these data is still being analysed.

Activity 8: Development of an RNN who classifies ambiguous stimuli presented in correlated sequences.
We have used Recurrent Neural Networks (RNN) to develop a network model that can perform the classification tasks that our rats perform, and which develops the same trials history biases observed in the experiments. We have found the following results: RNNs with single-gate units can be trained using the AC3 algorithm are able to learn the task and develop history transition biases. Transition biases were different for after-correct and after-error trials (panel XX) and are positively correlated with the probability of repetition. When increasing the size of the repeating-probability blocks, the magnitude of the biases also increase and the difference between the after-correct and after-error biases vanishes. Networks learn to rapidly adapt to changes in the repeating probabilities. 75% of the units in the network display activities that correlate with the number of repetitions occurred during the last 5 trials.

Activity 9: Discrimination experiments in head-fixed mice (Aim 1.4). We have develop a version of the same auditory discrimination task in head-fixed mice. The task in mice has a similar design as the version in rats except that mice lick from one of two ports places in front of the snout. Moreover, the lickports can be retracted and moved forward by means of a servomotor converting the task into a fixed stimulus duration paradigm. In addition to investigate the history biases developed by mice performing the task, we can investigate the dependence of those history biases on the introduction of a delay period between stimulus offset and the response.
1. We have developed a high-throughput open behavioral training platform with features that go beyond the state of the art: (1) following the Do it yourself (DIY) and open science philosophy we have designed behavioral boxes and acoustic isolation boxes that meet commercial standards for a much lower budget and most importantly that are just tailored to accommodate our needs. (2) they are all based on open-source software (Linux and Python) and all the codes are available (e.g. PyBPod). (3) they allow minimal human intervention during the training, reducing stress in animals and maximizing reproducibility of the data.

2. We have modelled the history biases of rats during a 2AFC task much beyond the state of the art. In particular we have identify different types of biases which are often confused, and set methods to isolate them and interpret them. We have been the first to separate the estimated probabilities animals may have about future events from the impact those predictions may have on future choices (i.e. in our case via chaice biases).

3. We have exploited the analysis of the movement of the animals during the task using novel published methods (e.g. DeepCut). We have been able to extract much more information from the behavior of the animals than standard methods based on nose pokes or tracking the overall position of the animal. For instance, we are now able to identify trials in which animals made changes of mind (CoM) by characterizing the fine orienting movements of their snouts towards one side port or the other. We are currently training deep convolutional networks to try to identify choice biases in the body position when they start a new trial (before taking the decision). Preliminary results show that animals “embody” many of these prediction strategies meaning that their body movements are different for instance when they are expecting a repeating stimulus from an alternating stimulus. We think that the combination of video analysis methods with the behavioral modeling and the electrophysiology will be a very powerful tool to understand the mechanisms underlying expectation and choice biases.

4. Our electrophysiology data from dorso-medial striatum has put forward the role of the basal ganglia in decision making and particularly in mediating the predictions and choice biases computed based on previous trials and maintained in memory during several trials in order to condition future choices. We have found several different types of neuronal population which, to our knowledge, had not been described. We are currently investigating the dynamics of these variables from a population perspective and will try to come up with network models that can subserve those computations observed in the DMS.

5. Our pharmacological results showing that medial Prefrontal Cortex (mPFC) and posterior parietal cortex (PPC)

The expected results for the second half of the project are:

1. Understand the extent to which the choice biases characterized so far are adaptive to our task. In oder words, why are animals adopting these expectation strategy? Can they modify the strategy in a context dependent manner? Can they adapt to more complex sequences (e.g. third order correlations in the stimulus sequence)?

2. Understand the urgency signals behind the timing of the responses in the reaction-time version of our task. This will require both behavioral controls (e.g. remove the perceptual decision and make just a timing task) and analysis of the electrophysiological data we have from basal ganglia and premotor areas (e.g. M2 area).

3. Be able to train Recurrent Neural Networks to solve out expectation-based 2AFC task and investigate the circuit mechanism that the trained RNNs use to form and maintain expectations as well as to combine them with the ambiguous sensory stimulus.

4. Perform optogenetics inactivation of DMS and mPFC during brief periods of the task to see how the different biases can be affected.

5. Be able to fit an individual drift-diffusion model to each subject (rodent or human) and use the dynamics of the decision variable derived from this model, to interrogate (1) the MEG data we have access to from our collaborator Prof. Tobias Donner or the (2) ephys data from premotor area M2.

6. Be able to reveal the neural basis of Changes of Mind from the combination of video analysis of single orienting movements together with the analysis of single trial neural populations. Expand our model of sensory integration and action selection to account for these changes of mind.

7. Investigate the generality of the expectation dynamics found in rats with a group of mice. See if the modelled developed for rat behavior can be fitted to mice data. Determine how sequential history effects are modulated by the introduction of a delay period and a Working Memory component. Finally, perform some wide-field of view Calcium imaging recordings in prefrontal areas of the mouse cortex. Determine whether the activity of any of these areas correlates with the history biases identified by the behavior model.