Skip to main content

MirrorBot: Biomimetic multimodal learning in a mirror neuron-based robot

Objective

We will develop an approach of biomimetic multimodal learning using a mirror-neuron-based robot to investigate the task of foraging for objects. This task involves the search for objects and integrates multimodal sensory inputs to plan and guide behaviour. We plan to examine these perceptual processes using models of cortical assemblies and mirror neurons to explore the emergence of semantic representations of actions, percepts, language and concepts in a MirrorBot, a biologically-inspired neural robot. The hypothesis under investigation is whether a mirror neuron-based cell assembly model will produce a life-like perception system for actions. We will develop an approach of biomimetic multimodal learning using a mirror-neuron-based robot to investigate the task of foraging for objects. This task involves the search for objects and integrates multimodal sensory inputs to plan and guide behaviour. We plan to examine these perceptual processes using models of cortical assemblies and mirror neurons to explore the emergence of semantic representations of actions, percepts, language and concepts in a MirrorBot, a biologically-inspired neural robot. The hypothesis under investigation is whether a mirror neuron-based cell assembly model will produce a life-like perception system for actions.

OBJECTIVES
1: To collect behavioural, imaging, and neural recording data from humans and non-human primates to clarify the neural perceptual processing;
2: To identify a realistic neural architecture for the MirrorBot robot to process perceptual visual and language data and generate actions;
3: To develop cell assemblies implementing the mirror neuron concept;
4: To integrate the cell assemblies to produce the behaviour in the MirrorBot;
5: To train and evaluate the MirrorBot.

DESCRIPTION OF WORK
Recently new theories and experiments in neuroscience have indicated that a biological and neuroscience-oriented approach for multimodal processing will lead to new life-like perception action systems. In particular, we will explore and further develop neuroscience evidence on mirror neurons or cell assemblies for life-like perception systems. These concepts from mirror neurons suggest that own actions, observed actions and language are very much interrelated since the same mirror neurons fire at the same time based on experiments with monkeys and humans. This project MirrorBot will develop and study emerging embodied representations based on mirror neurons. We will develop new techniques including cell assemblies, associative neural networks, and Hebbian-type learning in order to associate visual, auditory and motor concepts. We develop biomimetic multimodal learning and language instruction in a robot to investigate the task of foraging for objects. This task involves the search for objects while moving and integrating multimodal visual and language inputs. We will examine the emergence of representations of actions, perceptions, concepts, and language in a MirrorBot, a biologically inspired neural robot equipped with a polymodal associative memory.

Neuroscientific research indicates that mirror neurons have a critical role in cortical networks establishing links between perception, action and language. Mirror neuron areas correspond to cortical areas which are related to human language centres (e.g. Broca region) and could provide a cortical substrate for the integration of vision, language and action. A fundamental breakthrough in learning actions based on perceptual mirror-based representations could be of large benefit for intelligent assistants and service robots in the future.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

UNIVERSITY OF SUNDERLAND
Address
Langham Tower, Ryhope Road
SR2 7EE Sunderland
United Kingdom

Participants (4)

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
France
Address
Domaine De Voluceau
78153 Le Chesnay
MEDICAL RESEARCH COUNCIL
United Kingdom
Address
Park Crescent 20
W1B 1AL London
UNIVERSITA DEGLI STUDI DI PARMA
Italy
Address
Via Universita 12
43100 Parma
UNIVERSITAET ULM
Germany
Address
Gruener Hof 5C
89073 Ulm