Skip to main content

Vision Systems for a Natural Human Environment

Objective

The INSIGHT Action aimed at a greater understanding of the following topics:
-Primitives for representing three-dimensional surfaces underlying complex, local spatio-temporal and higher-order spatial operations.
-Computational procedures using these local operations that can be extracted from image sequences.
-The flexible and dynamic combination of the various three-dimensional primitives to yield three-dimensional representations and to segment images.
-The generation of three-dimensional surface models, and the use of prediction to integrate these into object models, allowing the prediction of motor action effects and assistance in object and scene identification.
The perspecitves of neuroscience, psychophysics, brain theory, mathematics and computer vision research are used to study vision processing and develop new ideas on the functioning of vision systems able to operate in a human environment, rather than a simple building block world.

Several novel experimental techniques have been developed: a versatile set up for visual stimulation at 100 Hz physiological experiments: a new stereometer for psychophysical study of ground planes; stimuli for cycloconvergent eye movements in response to disparity; set up for determining accuracy of optic flow estimates; and a new psychophysical technique to study shape from shade.

Results have been obtained on nonlinear interactions of optic flow components in cortical cells, cue invariance of shape representations in cortical neurons, and modelling of cortical cell responses to optic flow componnents. In addition, psychophysical data on texture stereo interaction has been obtained, as well as theoretical progress and experimental data on the use of disparity and its first and second order derivatives in perception of 3-dimensional surfaces; data on the interaction and differences between motion parallax and stereoscopic systems.

Theoretical results have been obtained on the affine second order optic flow, computation of 3-dimensional motion, estimation of local surface orientation using input from orientation selective filters similar to simple cell from mammalian visual cortex, on scale dependent differentiation and representations in scale space. Different methods for estimation of optic flow at the theoretical and practical level have been compared.

Methods have been developed and implemented to compute geometric characteristics of contours, to detect corners, to occlude contours and compute shape characteristics of the 3-dimensional surface near the rim.
APPROACH AND METHODS
Research activities were grouped into four themes:
-Higher-order local operations, studied anatomically and physiologically in the primate brain and psychophysically in human vision. Optic flow processing is receiving special attention, with the recovery of optic flow and its use in computer vision inves tigated under different assumptions.
-Combination of multiple depth cues, studied mainly from a psychophysical and computational point of view. The main topics are fusion of depth using cues from texture and stereo, and of structure using cues from motion.
-Cue combination for image segmentation, studied from a neurophysiological and theoretical point of view. This research is centred on texture segmentation and segmentation by motion.
-Surface and object representation, studied mainly from the computer vision point of view.
PROGRESS AND RESULTS
Several novel experimental techniques have been developed: a versatile set-up for visual stimulation at 100 Hz for physiological experiments; a new stereometer for psychophysical study of ground planes; stimuli for cycloconvergent eye movements in response to disparity; set-up for determining accuracy of optic flow estimates; and a new psychophysical technique to study shape from shade.
Results have been obtained on non-linear interactions of optic flow components in cortical cells, cue invariance of shape representations in cortical neurons, and modelling of cortical cell responses to optic flow components. In addition, psychophysical data on texture-stereo interaction has been obtained, as well as theoretical progress and experimental data on the use of disparity and its first and second order derivatives in perception of 3D surfaces; data on the interaction and differences between motion parallax and stereoscopic systems.
We have obtained theoretical results on the affine second order optic flow, computation of three dimensional motion, estimation of local surface orientation using input from orientation selective filters similar to simple cell from mammalian visual cortex, on scal dependent differentiation and representations in scale-space. Different methods for estimation of optic flow at the theoretical and practical level have been compared.
Methods have been developed and implemented to compute geometric characteristics of contours, to detect corners, to occlude contours and compute shape characteristics of the 3D surface near the rim.
POTENTIAL
The INSIGHT Action is improving and formalising collaboration between the disciplines concerned with understanding complex vision systems, and is already contributing to the development of new ideas on the functioning of vision systems able to operate in a human environment, as opposed to a simple "building-block" world. In the long run, this Action is expected to pave the way for the development of artificial vision systems able to operate in a changing three-dimensional world.

Coordinator

KATHOLIEKE UNIVERSITEIT LEUVEN
Address
Herestraat, 39
3000 Leuven
Belgium

Participants (12)

ECOLE POLYTECHNIQUE
France
Address
Route De Saclay
Palaiseau
Institut National de Recherche en Informatique et en Automatique - INRIA
France
Address
Domaine De Voluceau-rocquencourt
78153 Le Chesnay
KATHOLIEKE UNIVERSITEIT LEUVEN
Belgium
Address
De Croylaan, 52B
3030 Heverlee
ROYAL INSTITUTE OF TECHNOLOGY
Sweden
Address

100 44 Stockholm
RUHR-UNIVERSITY BOCHUM
Germany
Address
Universitätsstraße 150
44780 Bochum
UNIVERSITEIT VAN UTRECHT
Netherlands
Address
Princetonpleinweg 5, 80000
3508 TA Utrecht
UNIVERSITÄT KARLSRUHE
Germany
Address
Kaiserstraße 12
76131 Karlsruhe
Birkbeck College, University of London
United Kingdom
Address
Malet Street, Bloomsbury
WC1E 7HX London
University of Keele
United Kingdom
Address

ST5 5BG Keele
University of Sheffield
United Kingdom
Address
Western Bank
S10 2TN Sheffield
University of Stirling
United Kingdom
Address

FK9 4LA Stirling
Università degli Studi di Genova
Italy
Address
Via Dodecaneso 33
16146 Genova