Objective
We will develop an approach of biomimetic multimodal learning using a mirror-neuron-based robot to investigate the task of foraging for objects. This task involves the search for objects and integrates multimodal sensory inputs to plan and guide behaviour. We plan to examine these perceptual processes using models of cortical assemblies and mirror neurons to explore the emergence of semantic representations of actions, percepts, language and concepts in a MirrorBot, a biologically-inspired neural robot. The hypothesis under investigation is whether a mirror neuron-based cell assembly model will produce a life-like perception system for actions. We will develop an approach of biomimetic multimodal learning using a mirror-neuron-based robot to investigate the task of foraging for objects. This task involves the search for objects and integrates multimodal sensory inputs to plan and guide behaviour. We plan to examine these perceptual processes using models of cortical assemblies and mirror neurons to explore the emergence of semantic representations of actions, percepts, language and concepts in a MirrorBot, a biologically-inspired neural robot. The hypothesis under investigation is whether a mirror neuron-based cell assembly model will produce a life-like perception system for actions.
OBJECTIVES
1: To collect behavioural, imaging, and neural recording data from humans and non-human primates to clarify the neural perceptual processing;
2: To identify a realistic neural architecture for the MirrorBot robot to process perceptual visual and language data and generate actions;
3: To develop cell assemblies implementing the mirror neuron concept;
4: To integrate the cell assemblies to produce the behaviour in the MirrorBot;
5: To train and evaluate the MirrorBot.
DESCRIPTION OF WORK
Recently new theories and experiments in neuroscience have indicated that a biological and neuroscience-oriented approach for multimodal processing will lead to new life-like perception action systems. In particular, we will explore and further develop neuroscience evidence on mirror neurons or cell assemblies for life-like perception systems. These concepts from mirror neurons suggest that own actions, observed actions and language are very much interrelated since the same mirror neurons fire at the same time based on experiments with monkeys and humans. This project MirrorBot will develop and study emerging embodied representations based on mirror neurons. We will develop new techniques including cell assemblies, associative neural networks, and Hebbian-type learning in order to associate visual, auditory and motor concepts. We develop biomimetic multimodal learning and language instruction in a robot to investigate the task of foraging for objects. This task involves the search for objects while moving and integrating multimodal visual and language inputs. We will examine the emergence of representations of actions, perceptions, concepts, and language in a MirrorBot, a biologically inspired neural robot equipped with a polymodal associative memory.
Neuroscientific research indicates that mirror neurons have a critical role in cortical networks establishing links between perception, action and language. Mirror neuron areas correspond to cortical areas which are related to human language centres (e.g. Broca region) and could provide a cortical substrate for the integration of vision, language and action. A fundamental breakthrough in learning actions based on perceptual mirror-based representations could be of large benefit for intelligent assistants and service robots in the future.
Fields of science
Call for proposal
Data not availableFunding Scheme
CSC - Cost-sharing contracts
Coordinator
SR2 7EE Sunderland
United Kingdom
See on map
Participants (4)
78153 Le Chesnay
See on map
W1B 1AL London
See on map
43100 Parma
See on map
89073 Ulm
See on map