Skip to main content

Robots learning about objects from externalized knowledge sources

Objective

While today’s robots are able to perform sophisticated tasks, they can only act on objects they have been trained to recognize. This is a severe limitation: any robot will inevitably face novel situations in unconstrained settings, and thus will always have knowledge gaps. This calls for robots able to learn continuously about objects by themselves. The learning paradigm of state-of-the-art robots is the sensorimotor toil, i.e. the process of acquiring knowledge by generalization over observed stimuli. This is in line with cognitive theories that claim that cognition is embodied and situated, so that all knowledge acquired by a robot is specific to its sensorimotor capabilities and to the situation in which it has been acquired. Still, humans are also capable of learning from externalized sources – like books, illustrations, etc – containing knowledge that is necessarily unembodied and unsituated. To overcome this gap, RoboExNovo proposes a paradigm shift. I will develop a new generation of robots able to acquire perceptual and semantic knowledge about object from externalized, unembodied resources, to be used in situated settings. As the largest existing body of externalized knowledge, I will consider the Web as the source from which to learn from. To achieve this, I propose to build a translation framework between the representations used by robots in their situated experience and those used on the Web, based on relational structures establishing links between related percepts and between percepts and the semantics they support. My leading expertise in machine learning applied to multi modal data and robot vision puts me in a strong position to realize this project. By enabling robots to use knowledge resources on the Web that were not explicitly designed to be accessed for this purpose, RoboExNovo will pave the way for ground-breaking technological advances in home and service robotics, driver assistant systems, and in general any Web-connected situated device.

Call for proposal

ERC-2014-STG
See other projects for this call

Host institution

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Address
Via Morego 30
16163 Genova
Italy
Activity type
Research Organisations
EU contribution
€ 1 084 873

Beneficiaries (2)

FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA
Italy
EU contribution
€ 1 084 873
Address
Via Morego 30
16163 Genova
Activity type
Research Organisations
UNIVERSITA DEGLI STUDI DI ROMA LA SAPIENZA

Participation ended

Italy
EU contribution
€ 411 404
Address
Piazzale Aldo Moro 5
00185 Roma
Activity type
Higher or Secondary Education Establishments