Final Report Summary - CORMEL (Cortical Mechanisms of Exploratory Learning) Description of the Project ObjectivesIn this project, we addressed the following three goals:1. To establish a causal link between the frontopolar cortex (FPC) and exploratory learning in healthy participants. To this end, we applied transcranial direct current stimulation (tDCS) to the FPC in an experiment with a double-blind, within-subject design that used a well-validated behavioral learning paradigm. Exploratory learning was identified using Bayesian models of learning.2. To identify distributed patterns of brain activity underlying exploratory learning in healthy participants. To this end, we conducted an experiment using functional magnetic resonance imaging (fMRI) and the behavioral learning paradigm from (1). Of particular interest were brain regions that have been demonstrated or hypothesized to play a role in exploratory learning, namely, FPC, mid-intraparietal sulcus (IPS), ventral premotor cortex, the striatum, and the basal ganglia. The data were analyzed with partial least squares (PLS).3. To establish computationally and biologically interpretable models of the effects of tDCS to the FPC on exploratory learning behavior, neural sensitivity of FPC to reward-related parameters, and the strength and direction of interactions among neural regions supporting exploratory learning in healthy participants. To this end, we conducted an experiment with a double-blind, within-subject design that used the behavioral learning paradigm from (1) and simultaneous tDCS-fMRI. TDCS was used to systematically probe the neural mechanisms of exploratory learning by either up- or downregulating the FPC in each of the participants.Description of the Main Results Achieved and ConclusionsIn this section, we present the project’s results and conclusions against each of the project’s objectives:1. FPC activity has been associated with decisions to explore rather than exploit, but it is unclear whether its brain activity is indeed causally involved in steering the behavior. We investigated this with tDCS of the FPC. Seventy-nine participants performed a bandit task while undergoing anodal, cathodal, or sham placebo tDCS of the right FPC. Using Bayesian models of learning, we showed that participants engaged in more exploratory behavior under anodal stimulation (which enhances neural excitability) and less exploratory behavior under cathodal stimulation (which reduces neural excitability). In addition, anodal tDCS caused participants to explore more after recent unexpected losses and rendered them less focused on rewards when making decisions while performing the bandit task. This shows that activity in right FPC is indeed critical for exploratory behavior and that it integrates information about rewards and experiences to seek out alternative courses of action. These results were published in the Journal of Neuroscience in 2015 (see attached publication for further details).2. Using fMRI and PLS, we identified a distributed network of regions found to be more active for exploratory than exploitative choices (including bilateral FPC, left IPS, anterior cingulate, precuneus, and middle frontal gyrus) and another distributed network more active for exploitative than exploratory choices (including ventral medial prefrontal cortex, ventral premotor areas, and hippocampus). The interplay between these two networks may drive switching from a more immediately rewarding option to trying a new option with a less certain outcome. However, this fMRI can only identify correlations, it did not establish a causal link between changes in neural activity in these regions and behavior. We thus continued with (3).3. Finally, we conducted an experiment using simultaneous tDCS-fMRI to test the causal effects of right FPC activity on exploration and exploitation. Here, too, we were able to show that, following cathodal stimulation of right FPC, participants engaged in less model-based exploratory behavior. However, we could not replicate our previous anodal stimulation results: Instead, participants receiving anodal stimulation also engaged in less exploratory behavior relative to the sham stimulation group. Furthermore, a multivariate PLS analysis indicated that participants undergoing both anodal and cathodal stimulation showed decreased activity in bilateral frontopolar areas, ventral medial prefrontal cortex, globus pallidus, and inferior temporal areas during exploration relative to sham stimulation. While it was surprising to find decreased neural activity in areas targeted with anodal stimulation, these results establish a causal link between a right FPC-centered neural network and exploratory behavior, which had previously not been shown. We plan to further probe the strength and direction of the interactions among these regions involved in exploratory learning using dynamic causal models as well as measure the effects of stimulation on changes in FPC responses to reward-related parameters derived from the Bayesian modeling, and then disseminate the results of this study in a peer-reviewed journal.Socioeconomic ImpactThe results of our experiments have a significant socioeconomic impact: Our experiments reveal a causal biological mechanism underlying exploration, a fundamental aspect of human behavior that is necessary for survival and progress. This contributes to basic scientific knowledge and also provides potential opportunities for remediating clinical neuropathologies associated with lack of exploration (such as compulsivity and depression). Thus, as a result of the CORMEL project, we have begun collaborations with research groups with which we plan to evaluate changes in exploration after pharmacological manipulation of neural activity in healthy individuals, as well as in patients with depression. We have also started an experiment in collaboration with another local university (ETH Zurich) in which we investigate the effects of incidental emotions on exploratory behavior. The results of these experiments could provide avenues for the diagnosis and treatment of deficits in exploratory behavior in healthy and clinical populations.
