Package for New Operational Autostereoscopic Multiview systems and Applications
The ultimate goal for future telecommunication is highly effective interpersonal information exchange. The effectiveness of telecommunication is greatly enhanced by 3-D telepresence. In this concept it is crucial that visual information is presented in such a way that the viewer is under the impression of actually being physically close to the party with whom the communication takes place. Existing systems realise 3-D telepresence by stereoscopic imaging and display technologies. A natural 3-D impression is achieved if the camera positions correspond to the observers eye position and if the observer is stationary.
For telecommunication applications the need for glasses has to be avoided by applying an autostereoscopic display. In PANORAMA a multiviewpoint autostereoscopic system providing 3-D telepresence will be developed. In order to provide multiple views, the 3-D geometry of the captured scene will be estimated by means of image analysis. The 3-D geometry will be transmitted and images will be synthesised with respect to the observers eye positions. The PANORAMA project will perform research in the areas of hardware and software development of a multiviewpoint autostereoscopic system towards the use in telecommunication. Besides the usage for 3-D visual communication, several other 3-D imaging applications can benefit from the proposed system, due to the overlapping of many components. The additional applications are medicine, 3-D program production and automation in 3-D-modelling of industrial environments.
Consequently, this project is based on a thorough analysis of the requirements of each of the applications, provided by the interested partners. Therefore the unifying objective of the project is to:
Overcome current limitations in efficient communication that aim at telepresence.
This objective will be realised by enhancing and testing application systems for the capture, coding, transmission and presentation of multiview camera signals, based on the special software and hardware to be built in the project.
In the PANORAMA project two multiviewpoint autostereoscopic systems will be developed providing 3-D telepresence. One system will be focused on high quality, using off-line image analysis and fast, state-of-the-art graphics computers for the image synthesis (WPG1). The other system will be focused on real-time imaging for which special hardware for the image analysis as well for the coding of analysis data and the interpolation of intermediate viewpoints will be designed and developed (WPG2). Both systems will use a trinocular camera, an autostereoscopic display and the headtracking system to be developed in WPG3. Two different applications will be demonstrated in WPG4. These applications will make use of the software and hardware developments in the project and adapt the single parts to their requirements. These applications are »3-D Telepresence in Visual Communication«, where an advanced demonstrator will be realised and »Medicine«, where operational field trials with a clinical partner will be carried out.
The approach developed in WPG1 will use a parametric 3-D scene description in order to model the scene. This scene description is composed of model objects defined by their 3-D shape, 3-D motion and surface texture parameters. Each real object, like a person with a flexible surface or a table with a rigid surface, is represented by a model object. The parameters of the model objects will be estimated from trinocular input image sequences by means of image analysis. It is expected that the use of 3-D Model Objects will give a higher quality than using pixel-based methods. Also content based manipulations will be possible using a parametric 3-D scene description. Whereas the image synthesis can be carried out in real-time on available graphics computer, image analysis will be carried out off-line in software.
In order to be able to demonstrate a 3-D telepresence system in real-time, the necessary hardware will be developed in WPG2. The idea here is to generate intermediate views based on a disparity vector field with pixel accuracy. This way a look-around capability can be offered to the viewer: depending on the viewer«s position, the necessary views will be generated and displayed on the autostereoscopic display, where they can be viewed without glasses.
The trinocular camera and the autostereoscopic display will be built based on available technologies. The headtracking unit will be implemented in software, it will be able to return the viewer«s position to the system in real-time. This position will be input for the mechanical adjustments of the display and the generation of the necessary views.
Summary of Trial
All the developed Software and Hardware of PANORAMA will be used in two field trials: Video Communication and Medicine. In both field trials users will evaluate the advantages of the PANORAMA system compared to "traditional" imaging systems.
To provide image analysis methods and software libraries for the initialisation, handling, motion, shape and texture update of 3-D model objects for off-line image analysis,
To integrate all these methods into a single image analysis reference system,
To evaluate, specify and realise the subsystems for disparity estimation, disparity vector field coding and calculation of intermediate views according to the hardware needs,
To set up a hardware demonstrator for 3-D telepresence in visual communication, including a three channel camera, a real-time transmission over National Hosts, an autostereoscopic display and a real-time system that is able to detect the users head and eye position,
To demonstrate the application also in the fields of medicine and industrial inspection.
Software and Hardware will be developed in the project, aiming at new possibilities of reaching telepresence in future communications systems. In the Software part of PANORAMA new 3-D Modelling approaches will enable the modelling of real communication scenes without major restrictions. Using 3-D models will allow a good visual quality in the system making use of already available commercial software packages for the image synthesis at the receiver side. As an »intermediate« result, the Hardware development will allow the building of a real-time system. This will be achieved by building special designated Hardware for the Interpolation of Intermediate views using the disparity information of a stereoscopic signal. Another major point is the development of a European camera-system and a European autostereoscopic display. All these Software and Hardware activities will increase the European competitiveness in this promising area of telepresence.
All the developed components in the project will be used to carry out different field trials, involving expert and non-expert user groups. The two field trials planned in PANORAMA are »3-D Telepresence in Visual Communication« and »Medicine«. Both trials will return valuable feed-back about the acceptance of such a system and the necessary further developments to make such an autostereoscopic multiview system with look-around effect a success in the market.
Through enhancing and testing application systems for the capture, coding, transmission and presentation of multiview camera signals, based on the special software and hardware to be built in the project, the new system will overcome current limitations in efficient communication that aim at telepresence.
ZT IK 2
Tel: +49 89 636 53525
Fax: +49 89 636 52393