Periodic Reporting for period 3 - DREAM (Deferred Restructuring of Experience in Autonomous Machines)
Période du rapport: 2017-07-01 au 2018-12-31
Accumulating knowledge over long periods of time requires a consolidation process, so as to avoid being overwhelmed by the abundance of incoming information. Sleep has been shown to be critical for many consolidation processes, such as restructuring of representations, maintaining knowledge integration and coherence, improving insight learning, driving abstractions, forming novel levels of description, deleting unwanted information, exploring recombination of concepts, and stimulating creative thinking (Wagner et al., Nature, 2004). Our targeted scientific breakthrough is to enable robots to gain an open-ended understanding of the world over long periods of time, with alternating periods of experience and sleep. The possible benefits of sleep has so far been neglected in robotics and artificial intelligence.
To achieve higher levels of autonomy and understanding in developmental robotics, we propose a paradigm shift with DREAM, a cognitive architecture that exploits sleep to improve its functioning. It is contended here that Evolutionary methods (Fernando et al, Frontiers in Comp Neuro, 2012; Bellas et al., IEEE-TAMD, 2010) are a unifying principle for creative thinking and knowledge consolidation; these methods form the core of DREAM. Our key insight is that the brain consists of three coupled subsystems that are generated and adapted according to experience through evolutionary means: Models to make predictions about future state of the environment, notably to understand the results of actions; Policies that generate actions and behaviors, and are related to task-specific perceptual features; Values to reward, evaluate and compare policies or models. The long-term vision is to build genuinely situated and embodied agents with beliefs, desires, personalities, and idiosyncrasies, who are as inevitably influenced by their individual developmental trajectories as we are. To reach the proposed adaptive properties, the architecture will rely on alternating between active interaction and passive introspection over past events, i.e. sleep.
The cross-fertilization approach between robotics and neuroscience may have an impact that goes beyond engineering and robotics. We aim at better understanding how a representation restructuring process occurs, to provide robots with this ability. Animals and humans have this ability. Drawing inspiration from the knowledge we have of this process may then help us in our quest. The idea we have proposed to test in a robotics context may also help to better understand how animals and humans actually do it. Little is known yet about the neural substrate of this process. The algorithmic principles we are proposing may give some insights to neuro-scientists, just like reinforcement learning helped them build new models of reward-based learning.