Periodic Reporting for period 1 - MetaBot (Robotic embodiment of a meta-learning neural model of human decision-making)
Reporting period: 2018-05-01 to 2020-04-30
The project MetaBot was aimed at studying the neural and computational basis of reward-based meta-learning and decision-making in the human brain, by testing the effects of brain-body interaction during decision-making (embodiment) and the BOLD (fMRI) activity of the human brain during a decision-making task involving cost-benefit trade-off. A novel neuro-computational model (called RML, Silvetti et al., 2018), simulating neural dynamics for decision-making in humans (Figure 1), was developed and implemented in a robotic platform (iCub) for studying the capability of the RML to interact with the external environment through a humanoid body and then compared with human brain activity recorded by fMRI scanning during a decision-making task. In the task for the robotic platform, the iCub was asked to touch (reaching movement) one of two boxes placed in front of it. Each box, when touched, delivers a reward of variable magnitude and with variable probability (Figure 2). The iCub should discover and track over time which box is the most rewarding, making decisions through epochs when the reward magnitudes and probabilities do not change (stationary epoch) and epochs when they change for one or both the boxes (volatile epoch). The robot performs continuous decision-making operations, as during the reaching movement toward a box, it could change decision and select the other box. This process leads to specific arm trajectories that indicate the degree of decision uncertainty, so that low uncertainty decisions results in straighter arm trajectories, while high uncertainty decisions to more curved trajectories (see Spivey et al., 2005 and Lepora et al., 2015 for experimental results on humans). Finally, this decision-making process must take in consideration also the cost (in terms of motor energy expenditure) of changing decision (longer and inefficient reaching trajectories), so that the robot’s goal is to maximize reward while minimizing the motor effort required to execute the task. In the second part of the project, we tested the RML predictions on the brain activity from healthy volunteers, during a decision-making task performed during fMRI scanning. The task consisted in a decision epoch, where volunteers where asked to make a binary choice between an easy task (mental calculation) for a low reward and a harder task for a larger reward, testing different reward-difficulty combinations, and a performance epoch when the volunteers actually execute the mental calculation. Like for the iCub experiment, the decision-making process was aimed at maximizing reward while minimizing the cost of task execution. This project had a twofold objective. The first one (main one) was centered on providing new insights about the neurobiology of reward-based decision-making in the human brain, including the computational, anatomical and functional factors involved in it. We paid attention in particular to the role of brain-body interaction (embodied cognition), by simulating brain activity (through the RML model) in a humanoid robot performing a decision-making task. The second consisted in testing whether new algorithms, coming from computational neuroscience domain, could provide a new way for making robots cognition more flexible and reliable in decision-making tasks in ecological conditions (like humans do).
I acquired the necessary technical skills in humanoid robotics. Thanks to the help by the research group headed by Dr. Baldassarre, I have been able to get a good command of the field, although the width of technical knowledge needed to carry on the project resulted larger than expected. In the same period, I worked to conclude the development of the neural model RML. Finally, I created the bran-body interface connecting the iCub to the RML. I published a paper describing the RML model in the high impact journal “Plos Computational Biology”. Afterwards, I started working on the decision-making simulations with the iCub-RML system. When we were ready to analyze the simulations results, the project was paused due to the COVID-19 lockdown. Despite the end of the official financial support for MetaBot, we plan to conclude the analysis on the incoming autumn and submit the paper. At the same time, I started analyzing the fMRI data from healthy volunteers performing a decision-making task requiring the optimization of cost-benefit trade-off. These data were collected in collaboration with Ghent University. I conducted the data analysis following the “model-based” approach (O’Doerthy et al., 2007), a method designed for obtaining quantitative measures on how good a neuro-computational model is in predicting both the behaviour and the neural activity of human subjects. The preliminary results show that the RML predicts both the behaviour and the brain activity of human volunteers (Figure 3). These analyses are currently on going and I plan to submit a manuscript in autumn. Last but not least, during the progression of the MetaBot project, in collaboration with the research group of Dr Baldassarre and the Rome University Hospital “Campus Biomedico”, we obtained important results not explicitly predicted in the MetaBot project (incidental findings), nonetheless so relevant to be immediately communicated to the scientific community. These results are about the use of the RML software to clarify crucial aspects of Alzheimer’s Disease pathogenesis during its early stages. These findings were described in a manuscript now published in “Journal of Alzheimer’s Disease”. For concluding, the results deriving from MetaBot project have been reported also in popular science articles, one in the online journal Science Trends (https://sciencetrends.com/author/massimo-silvetti/) and the second in the blog hosted by the Donders Institute for Brain, Cognition and Behaviour (http://blog.donders.ru.nl/?p=8828&lang=en). Moreover, we communicated the results concerning the findings on the Alzheimer’s Disease to the press office of CNR, in order to obtain maximal visibility.
We achieved three main results that already projected us beyond the state of the art of neuroscience of decision-making: a) We published in Plos CB the RML model, showing that it is able to provide a unified theory on a plethora of experimental data about both neural and behavioural phenomena in decision-making in both human and nonhuman primates. b) The RML is showing to be effective in predicting new behavioural and fMRI data from healthy volunteers executing a decision-making task. This part of the project, close to be finished and submitted, is important for providing new insights on the neural basis of decision-making. c) The RML revealed to be crucial for providing new ideas and perspective on the early stages of Alzheimer’s Disease development (incidental results). These results are currently in press in the Journal of Alzheimer’s Disease and they will provide the starting point for developing a novel technology for the very precocious diagnosis of Alzheimer’s Disease (we are developing an ad hoc project). Finally, the results from the robotic implementation of the RML of the project will be useful for both neuroscience of decision-making (investigation of reward-based embodied decision-making) and they could lead to novel ideas on how to improve artificial intelligence algorithms for decision-making in autonomous robotics.