Community Research and Development Information Service - CORDIS

IBR augmentation

Techniques have been developed for visually augmenting real scenes with dynamic virtual objects. The techniques are applicable whether the real scenes are visualized to the user with Image-Based Rendering or simply by showing a live video stream of a real scene.

The developed techniques center around emulating the illumination conditions of the real scene when rendering the augmented virtual objects. If the virtual objects are not rendered with illumination conditions that are consistent with those of the real scene the augmented objects will stand out as conspicuous.

Specifically three different techniques addressing sub problems in this area have been developed. First, a framework for emulating the illumination conditions in a static scene has been developed. The approach applies equally well to indoor and outdoor scenery. The emulation is based on estimating parameters of a set of point light sources simulating the illumination conditions in the scene.

Secondly, a technique for relighting images of real scenes has been developed. The technique can completely alter the illumination conditions in images of a scene. The technique requires a full 3D model of the scene, and is therefore only applicable to scenes with man-made structures. Thirdly, a framework for estimating the changing illumination conditions of outdoor scenes has been developed. The framework makes it possible to model the illumination conditions from a video stream of a scene, responding correctly to changes in sun position, the colour of the sun and the skylight, and changes in cloud cover.

The main scientific result for all the three aspects is that they are capable of running in real-time on standard computers.

All the listed results are applicable to interactive, real-time augmented reality systems. It is envisages that augmented reality will be an important technology in many areas ranging from entertainment, over education to design prototyping and interactive apparatus repair.

Based on these results a prototype augmented reality system is being developed for show casing the technology to potential partners (research and/or commercial). The system will enable a user to visually explore a real scene by panning and tilting a flat panel screen mounted on a pole. The screen will show a video stream of the real world as well as arbitrary augmentations of for example historic building or events. The augmentations will look very close to photo-realistic since their illumination conditions will mimic those of the real scene.

Related information

Reported by

Niels Jernes Vej 14
See on map
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top