CORDIS
EU research results

CORDIS

English EN

SenseMaker: A Multi-sensory, Task-specific, Adaptable perception System

Project information

Grant agreement ID: IST-2001-34712

  • Start date

    1 June 2002

  • End date

    31 October 2005

Funded under:

FP5-IST

  • Overall budget:

    € 2 173 422

  • EU contribution

    € 1 606 000

Coordinated by:

UNIVERSITY OF ULSTER

United Kingdom

Objective

The aim of this project is to conceive and implement electronic architectures that are able to merge sensory information sampled through different modalities into a unified perceptual representation of our environment. The architectural design will be based on the biological principles of sensory receptor and nervous system function. At a higher cognitive level, this representation of the proximal environmental space will be largely independent of the selected sensory substrates. A major objective of the project would be to enable a system to reconfigure itself, forming supplementary cross-connections between the sensory receptor level of a given type and the higher stages of processing specific to another sensory modality. The ambition is to create new senses to supplement the existing senses e.g. allow one to see the thunder or hear the lightning.

DESCRIPTION OF WORK
The brain is able to extract correlated information from sensory representations elaborated simultaneously using different sensory modalities. The "sense maker" system will choose a minimal set of sensory modalities from a predefined library, the combination of which will lead to reliable object/environment discrimination and identification.
The natural senses to be considered for emulation are vision, audition, and hepatic senses, and internal representation of motor command. The composite perceptual system, to be investigated using neuromimetic modelling and implemented using programmable mixed-analogue digital ASICs, will adapt its computational architecture as a function of unpredicted changes in the environment, or when faced with the partial impairment of some sensors. The sense-maker system will be composed of several stages, modelled on biological architectural principles: The lower level will correspond to different sensory input layers, each associated with a single modality (visual, auditory, electro reception), working in parallel.
An intermediate stage will regulate input-output transfer within each modality channel, depending on the rate of change in the input message, motor/probing activity and the immediate need for resource allocation for internal processing. The artificial system to be designed and implemented would include an additional layer in order to bind and store, in a reversible manner, the most frequent cross-sensory associations and the most predictable context gathered from recent statistics taken from the environment. Associative plasticity rules would be inspired from biological spike-timing dependent plasticity algorithms studied in mammalian neocortex, and used to store long-term memories. The topmost layer will constitute the intelligent processor of the analyser, and decides how to distribute the focus of attention and hence computational power allocation.

Coordinator

UNIVERSITY OF ULSTER

Address

Cromore Road
Bt52 1sa Coleraine

United Kingdom

Participants (5)

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE

France

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE

France

ECOLE NATIONALE SUPERIEURE D'ELECTRONIQUE, INFORMATIQUE ET RADIOCOMMUNICATIONS DE BORDEAUX

France

RUPRECHT-KARLS-UNIVERSITÄT HEIDELBERG

Germany

THE PROVOST FELLOWS AND SCHOLARS OF THE COLLEGE OF THE HOLY AND UNDIVIDED TRINITY OF QUEEN ELIZABETH NEAR DUBLIN

Ireland

Project information

Grant agreement ID: IST-2001-34712

  • Start date

    1 June 2002

  • End date

    31 October 2005

Funded under:

FP5-IST

  • Overall budget:

    € 2 173 422

  • EU contribution

    € 1 606 000

Coordinated by:

UNIVERSITY OF ULSTER

United Kingdom

This project is featured in...