Skip to main content

CONVOLUTION AER VISION ARCHITECTURE FOR REAL-TIME

Objective

Biological brains are structured in layers of neurons, where neurons in a layer connect to a very large number of neurons in the following layer. Each neuron in a layer connects to a projective field in the next layer. This can be approximated by two-dimensional convolutions. For Real-Time solutions direct hardware implementations are required. But hardware engineers face a very strong barrier: the massive connectivity. The problem gets worst for multi-chip multi-layer hierarchical bio-like systems. Address-Event-Representation (AER) is an incipient bio-inspired spike-based technique capable of providing a hardware solution.

The objectives of CAVIAR are two-fold:
1) To develop a general AER infrastructure for constructing bio-inspired hierarchically structured multi-chip systems for sensing + processing + actuation;
2) The implementation of a particular perceptive-action demonstrator vision system exploiting this infrastructure.

OBJECTIVES
The objective of CAVIAR is to develop a robust AER infrastructure capable of supporting bio-inspired multi-chip multi-layer hierarchical sensing/processing/actuation systems. This will provide a unique platform to facilitate long-term research on and development of complex bio-inspired systems and to test novel ideas in spike-based processing. The following parts will be developed within the CAVIAR project: VLSI Chips: Sensing Retina Chip, Programmable-kernel Convolution Processing Chip, Dimension Reduction Competition 'Object' Chip, Spatio-temporal Learning Chip. AER Interfaces: Chip-to-Chip Interfacing Module, Chip-to-Computer Interfacing Module. Demonstrator: build a vision system for detecting balls and mount it on a robot. Bus Interactions and Consortium Standards: We will define a minimum set of AER standards that will let partners develop parts independently, such that it will be possible to interconnect them when assembling more complicated, hierarchical systems.

DESCRIPTION OF WORK
The work is structured in 7 WPs: WP1: Sensing Retina Chip. A medium-resolution retina chip will be provided that responds to local positive and negative irradiance transients in a two-dimensional input image. It performs focal-plane brightness adaptation and data compression for efficient use of the sensing and communication bandwidths;
WP2: Programmable-kernel Convolution Processing Chip. A processing chip will be provided that implements spatial filtering operations such as those found in biological neural systems. This chip will execute real-time convolution filtering operations on a two-dimensional data set. The convolution kernel will be programmable, such that a set of identical convolution chips can be used to extract different features at different spatial scales in parallel from a single data set;
WP3: Dimension Reduction Competition 'Object' Chip. We will develop a dimensionality-reduction and in-layer competition chip. It will combine several sparsely-coded two-dimensional maps, such as those provided by a set of convolution chips, select the dominant feature and output the coordinates of its centre of mass;
WP4: Spatio-temporal Learning Chip. A spatio-temporal pattern learning chip will be develop that performs unsupervised learning to recognize specific shapes, speeds and directions using a spike-based learning rule. This learning will help to predict the course of typical pattern sequences;
WP5: AER Interfaces. A set of chip-computer and chip-chip interfaces will be developed for testing, diagnosis, and development of hierarchical AER systems;
WP6: Demonstrator. To drive properly the development of the previous components the consortium will develop as well a demonstrator vision system in which all developed components will be put together on a robot platform for detecting and following a moving ball;
WP7: Bus Interactions and Consortium Standards. A set of common rules will be set, tested, and discusses for assembling AER components.

Call for proposal

Data not available

Coordinator

CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS
Address
C/ Serrano 117
28006 Madrid
Spain

See on map

Participants (3)

EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH
Switzerland
Address
Raemistrasse 101
8092 Zuerich

See on map

UNIVERSIDAD DE SEVILLA
Spain
Address
Calle S. Fernando 4
41004 Sevilla

See on map

UNIVERSITETET I OSLO
Norway
Address
Problemveien 5-7
0316 Oslo

See on map