Skip to main content

Reconstruction of Reality from Image Sequences

Objective

Virtual reality systems offer exciting possibilities. We propose to help construct virtual reality databases automatically from sequences of images taken by moving camera(s). Such sequences contain, a priori, the necessary information for being able to determine the 3D structure of a scene and its photometric aspects.

APPROACH AND METHODS

The principal objective is to define and develop algorithms for extracting data from sequences of images acquired with a slowly moving camera. We hope to define a sequence of operations such that the construction of the database from the sequences of images is as automatic as possible.

- Existing computer vision research shows that it is possible to extract the 3D structure of a scene which is observed by a stereovision setup or by a camera following an unknown path.

However, the 3D structure and the motion are note very precisely estimated and the consortium will explore new ideas to improve these results. It will also address the internal parameters of the camera changing during a sequence (zoom for example).

- To compute a realistic view of a scene, we must not only know the structure of the scene but also we must determine how the surfaces reflect light. The BRDF (Bidirectional Reflection Distribution Function) completely characterises surface properties at a given level of resolution. It is the most natural description of local surface properties. We will study how to estimate it from the sequences of images and from the estimated structure of the scene.
We will also work on 3D texture as a means to characterise the reflectance of materials at any scale.

- The information which could be obtained by analysing the sequences will probably be very different from the ones which are obtained usually by constructing databases from maps by an operator. We will explore how the data structures can be adapted to the requirements of virtual reality systems.

POTENTIAL

Our first result will be to provide new interesting insights in the three research areas that we address: structure from motion in computer vision, understanding of the reflectance properties of materials and synthesis of images from noisy and unstructured data. We aim to setup the following exciting demonstration: select an old film which includes sequences of images of a building which no longer exists and then create a database for it. Other possibilities include the construction of databases necessary for the simulation of driving cars, trains, boats, planes or for the design of furniture.

Coordinator

Thomson CSF
Address
Domaine De Corbeville
91404 Orsay
France

Participants (3)

Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung eV (FhG)
Germany
Address
Wilhelminenstraße 7
64283 Darmstadt
Institut National de Recherche en Informatique et en Automatique - INRIA
France
Address
Domaine De Voluceau-rocquencourt
78153 Le Chesnay
UNIVERSITY OF UTRECHT
Netherlands
Address
Heidelberglaan, 880 125
3584 CS Utrecht