Skip to main content

SenseMaker: A Multi-sensory, Task-specific, Adaptable perception System

Objective

The aim of this project is to conceive and implement electronic architectures that are able to merge sensory information sampled through different modalities into a unified perceptual representation of our environment. The architectural design will be based on the biological principles of sensory receptor and nervous system function. At a higher cognitive level, this representation of the proximal environmental space will be largely independent of the selected sensory substrates. A major objective of the project would be to enable a system to reconfigure itself, forming supplementary cross-connections between the sensory receptor level of a given type and the higher stages of processing specific to another sensory modality. The ambition is to create new senses to supplement the existing senses e.g. allow one to see the thunder or hear the lightning.

DESCRIPTION OF WORK
The brain is able to extract correlated information from sensory representations elaborated simultaneously using different sensory modalities. The "sense maker" system will choose a minimal set of sensory modalities from a predefined library, the combination of which will lead to reliable object/environment discrimination and identification.
The natural senses to be considered for emulation are vision, audition, and hepatic senses, and internal representation of motor command. The composite perceptual system, to be investigated using neuromimetic modelling and implemented using programmable mixed-analogue digital ASICs, will adapt its computational architecture as a function of unpredicted changes in the environment, or when faced with the partial impairment of some sensors. The sense-maker system will be composed of several stages, modelled on biological architectural principles: The lower level will correspond to different sensory input layers, each associated with a single modality (visual, auditory, electro reception), working in parallel.
An intermediate stage will regulate input-output transfer within each modality channel, depending on the rate of change in the input message, motor/probing activity and the immediate need for resource allocation for internal processing. The artificial system to be designed and implemented would include an additional layer in order to bind and store, in a reversible manner, the most frequent cross-sensory associations and the most predictable context gathered from recent statistics taken from the environment. Associative plasticity rules would be inspired from biological spike-timing dependent plasticity algorithms studied in mammalian neocortex, and used to store long-term memories. The topmost layer will constitute the intelligent processor of the analyser, and decides how to distribute the focus of attention and hence computational power allocation.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

UNIVERSITY OF ULSTER
Address
Cromore Road
BT52 1SA Coleraine
United Kingdom

Participants (5)

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE
France
Address
3, Rue Michel-ange
75794 Paris Cedex 16
CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE
France
Address
3, Rue Michel-ange
75794 Paris Cedex 16
ECOLE NATIONALE SUPERIEURE D'ELECTRONIQUE, INFORMATIQUE ET RADIOCOMMUNICATIONS DE BORDEAUX
France
Address
1, Avenue Du Docteur Albert Schweitzer
33402 Talence
RUPRECHT-KARLS-UNIVERSITÄT HEIDELBERG
Germany
Address
Seminarstraße 2
Heidelberg
THE PROVOST FELLOWS AND SCHOLARS OF THE COLLEGE OF THE HOLY AND UNDIVIDED TRINITY OF QUEEN ELIZABETH NEAR DUBLIN
Ireland
Address
College Green
2 Dublin