Periodic Reporting for period 2 - GOAL-Robots (Goal-based Open-ended Autonomous Learning Robots)
Reporting period: 2017-11-01 to 2019-04-30
Imagine your friend asks you to tidy up her kitchen which is full of furniture and objects. Now imagine that your friend described through a photo how the kitchen should look like after being tidy up. This task would be boring for you, but you would nevertheless be able to carry it out with ease. Indeed, when you were a child you played with all sort of objects and so, driven by curiosity, you learn many flexible sensorimotor skills to manipulate them at will. Differently from you, current robots are ill-suited for these type of challenges. Indeed, although new architecture and algorithms are developed to control robots, leading them to accomplish tasks in unstructured environments still requires much programming or supervised training. Moreover, in such current robots can become able to solve specific tasks but in comparison to biological agents they have a very limited autonomy and flexibility.
Why the project is important for society? For many applications, future robots should be able to learn how to solve multiple tasks in unstructured environments in an autonomous way. The project aims to build artificial intelligence architectures that should allow robots to self-generate goals and autonomously learn sensorimotor skills to accomplish them, based on the cumulative acquisition, modification, and recombination of previously acquired skills. This will allow robots to accomplish users’ tasks (e.g. to set a room in a desired state) with little additional learning.
What are the project objectives? The project follows a previous European project called IM-CLeVeR (“Intrinsically Motivated Cumulative Learning Versatile Robots”) which investigated how intrinsic motivations (IMs) can support autonomous learning in biological and artificial agents. IMs, related to novelty, surprise, and competence acquisition, are maximally apparent in children at play: driven by IMs, children explore and interact with objects in the environment thus getting to know how they work and to acquire a wide set of sensorimotor skills suitable to manipulate them as desired. GOAL-Robots central idea is that IMs can fully support open-ended learning only if a further critical ingredient is considered: goals. A goal is an internal representation of a state (or set of states, or trajectories of states) of the world that can be internally activated by the agent in the absence of its current perception: when a goal representation is internally activated, it can contribute to move the agent’s attention, behaviour, and learning resources towards the accomplishment of the goal. An example of goal is ""I would like to see that glass of water in my hand"" and an example of action to accomplish such goal is ""I will move my hand and grasp the glass"". An example of more complex goal is the one considered above ""I would like to tidy up the kitchen after dinner"". The project idea is that endowing robots with the capacity of self-generating goals, and using them to learn the skills for their accomplishment, is the key ingredient to have truly autonomous open-ended learning robots.
(1) The project has carried out a number of experiments with infants with the objective of understanding the mechanisms through which they undergo open-ended learning. We investigated in depth “contingency-based learning”, a mechanism at the core of infants’ learning, for which the consequences of actions, internally represented as goals, motivate the infant to refine the actions leading to them. This mechanism was also studied in depth in computational models shown to be able to learn how to produce effects on own body (e.g. to touch own body parts with a hand).
(2) A second achievement was the development of controllers allowing robots to self-generate goals to guide autonomous open-ended learning. The idea is that a robot is left alone in an environment and by exploring it with already-acquired actions it should self-generate goals. One way to do this is based on the idea to form goals as states that follow the robot's actions that cause changes in the environment, or based on prediction errors.
(3) The self-generated goals should then guide the robots to acquire skills accomplishing the goals: these skills are used to pursue goals that become desired in the future, e.g. to tidy up a kitchen. This requires architectures that integrate different functions, all implemented in the project robots: deciding on which goal to invest learning time; integrating the acquired skills; learning forward models; re-using the previously acquired goals and skills to accomplish the goals of the robot's users, e.g. based on planning.
(4) The project is also showing how the devised systems work in real robots. In particular, we have developed a robotic scenario to test the ability of robots for open-ended learning and then for re-using the acquired knowledge to accomplish externally assigned tasks, e.g. to put some objects back into their desired place. The task can have an increasing complexity, in particular the task can require to simply work on a table plane, or to also move objects on a shelf (which implies grasping the objects), or to configure the objects in a given relative state (e.g. to form a pile).