Periodic Reporting for period 5 - DYCOCIRC (Basal ganglia circuit mechanisms underlying dynamic cognitive behavior)
Reporting period: 2023-10-01 to 2024-09-30
Our project seeks to understand the mechanisms by which the brain generates dynamic, internal representations used for cognitive processes and how those representations are transformed into adaptive behavior. This is of fundamental importance, as internal representations are generated when we think and deliberate, when we plan ahead in pursuit of a goal, or call on past experience to inform our current course of action. Dysfunction in cognition or the translation of cognitive information into motor output characterizes a number of neuropsychiatric and neurological disorders, and designing artificial systems capable of the intelligent cognitive processing similar to that found in humans and animals is a major goal of engineers seeking to build the next generation of artificial intelligence. Our work has already led to new understanding relevant for understanding brain function in health and disease, as well as revealing insights that are relevant for AI. In our project, DYCOCIRC, we are leveraging cutting edge tools in circuit and computational neuroscience to understand how rodent model organisms transform internal information about elapsed time into a decision and ultimately motor output. By combining tools for monitoring and manipulating targeted populations of neurons with computational modelling and carefully designed decision-making behavior, our goal is to gain an algorithmic and neural level understanding of how circuits that involve the basal ganglia contribute to the generation of dynamic internal variables and to the execution of movements required for adaptive behavior.
Since the start of the project, we have completed five studies currently in various stages of publication, and that are posted to the pre-print server BioRxiv. First, by analyzing phasic dopamine activity in relation to decision-making behavior and computational reinforcement learning models, we discovered that mice use efficient, compressed representations of cognitive variables to perform the task, extending the principle of efficient coding to the cognitive domain.
Second, by combining fiber photometric recordings of activity from the projection cell types initiating the two major circuits of the basal ganglia, so called direct and indirect medium spiny neurons of the striatum (dMSN, iMSN) with a series of optogenetic inhibition experiments in the context of the aforementioned behavioral task, we have revealed a novel model of sensorimotor striatum wherein the direct pathway appears to be necessary for augmenting generalized movement vigor, and the indirect pathway for the proactive suppression of specific behaviors. These data reveal new insight into the etiology of neurological and neuropsychiatric diseases such as Parkinson’s, Huntington’s diseases, and ADHD that are characterized by varying degrees of dysfunction in suppressive control of behavior.
Third, we applied an unconventional manipulation, temperature, to striatal tissue to rescale striatal activity in time, and found we could rescale temporal decisions. In contrast, temperature did not affect low level timing of movements. These data demonstrate that the time-course of evolving striatal population activity dictates the speed of a latent timing process that is used to guide decision-making but that is not used to specify the details of movement execution, with broad implications for understanding both the neural basis of timing and, more generally, the role of basal ganglia circuits in behavior.
Fourth, we applied the principles of temporal scaling to an existing data set wherein stimulus magnitude was shown to alter the temporal judgments of rats, and discovered a tight link between the effects of stimulus magnitude on duration judgment and temporal scaling of neuronal activity.
Lastly, we developed new computational reinforcement learning theory to extend the concept of distributional reward coding to multiple dimensions, both time and magnitude, and discovered evidence that computations similar to that predicted by the theory are implemented in the brain by recording from optogenetically tagged midbrain dopamine neurons in mice.
All of this work has been communicated at scientific meetings including the annual Cosyne conferences, the, Society for Neuroscience meetings, the Ascona conference on neural circuits, a Janelia Farm workshop on foraging, a Brain prize meeting on neural dynamics, as well as many invited seminars spanning North and South America and Europe. It has also been discussed and written about extensively in both the national Portuguese media and Internationally, as well as on social media.
Second, by combining fiber photometric recordings of activity from the projection cell types initiating the two major circuits of the basal ganglia, so called direct and indirect medium spiny neurons of the striatum (dMSN, iMSN) with a series of optogenetic inhibition experiments in the context of the aforementioned behavioral task, we have revealed a novel model of sensorimotor striatum wherein the direct pathway appears to be necessary for augmenting generalized movement vigor, and the indirect pathway for the proactive suppression of specific behaviors. These data reveal new insight into the etiology of neurological and neuropsychiatric diseases such as Parkinson’s, Huntington’s diseases, and ADHD that are characterized by varying degrees of dysfunction in suppressive control of behavior.
Third, we applied an unconventional manipulation, temperature, to striatal tissue to rescale striatal activity in time, and found we could rescale temporal decisions. In contrast, temperature did not affect low level timing of movements. These data demonstrate that the time-course of evolving striatal population activity dictates the speed of a latent timing process that is used to guide decision-making but that is not used to specify the details of movement execution, with broad implications for understanding both the neural basis of timing and, more generally, the role of basal ganglia circuits in behavior.
Fourth, we applied the principles of temporal scaling to an existing data set wherein stimulus magnitude was shown to alter the temporal judgments of rats, and discovered a tight link between the effects of stimulus magnitude on duration judgment and temporal scaling of neuronal activity.
Lastly, we developed new computational reinforcement learning theory to extend the concept of distributional reward coding to multiple dimensions, both time and magnitude, and discovered evidence that computations similar to that predicted by the theory are implemented in the brain by recording from optogenetically tagged midbrain dopamine neurons in mice.
All of this work has been communicated at scientific meetings including the annual Cosyne conferences, the, Society for Neuroscience meetings, the Ascona conference on neural circuits, a Janelia Farm workshop on foraging, a Brain prize meeting on neural dynamics, as well as many invited seminars spanning North and South America and Europe. It has also been discussed and written about extensively in both the national Portuguese media and Internationally, as well as on social media.
Our work combines carefully designed behaviors that bring under control the inner workings of cognition and adaptive behavior, with cutting edge tools for monitoring and manipulating targeted populations of neurons as well as computational modeling to reveal the algorithms and neural mechanisms of cognition. Such an integrative approach, bringing together tools from experimental psychology, molecular biology, neurophysiology, AI and computer science, we believe will be critical for deepening our understanding of both natural and machine adaptive behavior as well as their failure modes.
In our 2022 paper published in Nature Neuroscience, Motiwala et al. leveraged the well-known computational function of phasic dopamine to signal a temporal difference reward prediction error, together with careful analysis of decision-making and reaction time data in mice to infer the underlying internal representation of cognitive variables mice used to perform a decision-making task.
In our 2022 paper published in Nature, Cruz et al. combined fiber photometric and electrophysiological recordings of direct and indirect pathway medium spiny striatal neurons with optogenetic inhibition experiments during the decision-making behavior with the construction of a computational reinforcement learning model that contained multiple features of the basal ganglia circuitry. In doing so, we discovered a new principle by which the parallel, push pull circuit of the basal ganglia functions to control behavior, with different components of behavioral control programs becoming embedded in distinct, regional and cell-type specific elements.
In our 2023 paper published in Nature Neuroscience, Monteiro, Rodrigues, Pexirra et al. developed a closed loop, thermo-electric device for focal manipulation of temperature in target brain regions. We provided arguably the strongest evidence to date of a core principle for neural circuit function, that the temporal evolution of neuronal population activity provides the time base for circuit computations.
We developed a new computational reinforcement learning theory, Time-Magnitude Reinforcement Learning (TMRL), extending recent developments in machine learning to allow for learning information about distributions of rewards. Distributional reinforcement learning has been shown to greatly improve performance of deep reinforcement learning agents on strategy games, and it was recently found that midbrain dopamine neurons, a core component of the brain’s reward circuitry, display evidence of distributional reward coding. By recording from optogenetically tagged dopamine neurons, we demonstrated that midbrain dopamine neurons display evidence of multi-dimensional distributional coding in line with TMRL.
In our 2022 paper published in Nature Neuroscience, Motiwala et al. leveraged the well-known computational function of phasic dopamine to signal a temporal difference reward prediction error, together with careful analysis of decision-making and reaction time data in mice to infer the underlying internal representation of cognitive variables mice used to perform a decision-making task.
In our 2022 paper published in Nature, Cruz et al. combined fiber photometric and electrophysiological recordings of direct and indirect pathway medium spiny striatal neurons with optogenetic inhibition experiments during the decision-making behavior with the construction of a computational reinforcement learning model that contained multiple features of the basal ganglia circuitry. In doing so, we discovered a new principle by which the parallel, push pull circuit of the basal ganglia functions to control behavior, with different components of behavioral control programs becoming embedded in distinct, regional and cell-type specific elements.
In our 2023 paper published in Nature Neuroscience, Monteiro, Rodrigues, Pexirra et al. developed a closed loop, thermo-electric device for focal manipulation of temperature in target brain regions. We provided arguably the strongest evidence to date of a core principle for neural circuit function, that the temporal evolution of neuronal population activity provides the time base for circuit computations.
We developed a new computational reinforcement learning theory, Time-Magnitude Reinforcement Learning (TMRL), extending recent developments in machine learning to allow for learning information about distributions of rewards. Distributional reinforcement learning has been shown to greatly improve performance of deep reinforcement learning agents on strategy games, and it was recently found that midbrain dopamine neurons, a core component of the brain’s reward circuitry, display evidence of distributional reward coding. By recording from optogenetically tagged dopamine neurons, we demonstrated that midbrain dopamine neurons display evidence of multi-dimensional distributional coding in line with TMRL.