The project MetaBot was aimed at studying the neural and computational basis of reward-based meta-learning and decision-making in the human brain, by testing the effects of brain-body interaction during decision-making (embodiment) and the BOLD (fMRI) activity of the human brain during a decision-making task involving cost-benefit trade-off. A novel neuro-computational model (called RML, Silvetti et al., 2018), simulating neural dynamics for decision-making in humans (Figure 1), was developed and implemented in a robotic platform (iCub) for studying the capability of the RML to interact with the external environment through a humanoid body and then compared with human brain activity recorded by fMRI scanning during a decision-making task. In the task for the robotic platform, the iCub was asked to touch (reaching movement) one of two boxes placed in front of it. Each box, when touched, delivers a reward of variable magnitude and with variable probability (Figure 2). The iCub should discover and track over time which box is the most rewarding, making decisions through epochs when the reward magnitudes and probabilities do not change (stationary epoch) and epochs when they change for one or both the boxes (volatile epoch). The robot performs continuous decision-making operations, as during the reaching movement toward a box, it could change decision and select the other box. This process leads to specific arm trajectories that indicate the degree of decision uncertainty, so that low uncertainty decisions results in straighter arm trajectories, while high uncertainty decisions to more curved trajectories (see Spivey et al., 2005 and Lepora et al., 2015 for experimental results on humans). Finally, this decision-making process must take in consideration also the cost (in terms of motor energy expenditure) of changing decision (longer and inefficient reaching trajectories), so that the robot’s goal is to maximize reward while minimizing the motor effort required to execute the task. In the second part of the project, we tested the RML predictions on the brain activity from healthy volunteers, during a decision-making task performed during fMRI scanning. The task consisted in a decision epoch, where volunteers where asked to make a binary choice between an easy task (mental calculation) for a low reward and a harder task for a larger reward, testing different reward-difficulty combinations, and a performance epoch when the volunteers actually execute the mental calculation. Like for the iCub experiment, the decision-making process was aimed at maximizing reward while minimizing the cost of task execution. This project had a twofold objective. The first one (main one) was centered on providing new insights about the neurobiology of reward-based decision-making in the human brain, including the computational, anatomical and functional factors involved in it. We paid attention in particular to the role of brain-body interaction (embodied cognition), by simulating brain activity (through the RML model) in a humanoid robot performing a decision-making task. The second consisted in testing whether new algorithms, coming from computational neuroscience domain, could provide a new way for making robots cognition more flexible and reliable in decision-making tasks in ecological conditions (like humans do).
