You are here : 
Signal Processing & Mixed Reality
Workshop on image processing and computer vision for mixed reality
Thursday, 13 March 2003
The goal of this workshop was to review techniques in the field of image processing and computer vision and their applications for mixed reality. A section of presentations that covered scientific aspects was followed by a panel discussion that covered practical implementation and user issues together with future directions.
The first speaker was Luc Robert (Realviz). He was addressing the issue that mixed reality must become an industrial reality to be successful. In his mind computer vision and image processing should support users in interactive tools. He gave some examples of his work in post-production/special effects and visualisation of architectural objects (projects STAR and CREATE).
Reinhard Koch (CAU, University of Kiel) was introducing the concept of plenoptic modelling. The main idea is to use a rig of cameras to acquire images from a scene. In an off-line process a self-calibration and dense depth-maps are computed for all images. Instead of aiming at a global consistent 3D model all the images and computed depth maps are used in online rendering. Examples of work done in ORIGAMI were presented.
It was addressed to the first two speakers that they should consider to contribute to MPEG, because multi-camera/images are currently and issue there.
Stefano Tubaro (Polytechnico di Milano) gave a presentation on 3D shape reconstruction from only one camera moving around objects. The technique developed within the ORIGAMI project is using a level-set approach. This allows an implicit representation of objects without the need of fixing the topology at the beginning. Results for some close-range objects were shown. Stefano pointed out the efficiency of their implementation (just a few seconds to converge).
Jochen Winbermuehle (University of Hanover) was presenting work done in PISTE. The aim of that project was to provide tools for the generation of 3D interactive content of sport events. The approach was a segmentation followed by fitting a simple 3D model and 3D motion estimation. Typically two cameras were used.
The last speaker of the presentation session was Luc van Gool (ETH Zurich and KU Leuven). He was talking about analysing facial expressions for applications in facial animation. The work was done in the MESH project. The approach was inspired by the work on face spaces. The used concept of 'visemes' represent 'atoms' of facial expressions; any expression can be composed with these.
After the presentations a panel discussion has taken place. The panelists were:
- Graham A. Thomas, BBC R&D, Virtual production in the broadcast environment
- Paul Walland, Snell&Wilcox, (MetaVision) The 'reality' of mixed reality
- Didier, Stricker, Fraunhofer IGD, (ARVIKA, Archeoguide), Industrial AR+VR applications
- Marc Op de Beeck, Philips Research Laboratories, (ATTEST), 3D for the home user
- Gauthier Lafruit, IMEC, Mixed reality in communications, Mixed reality in communications
Every participant gave an initial statement. G.Thomas showed a short documentary video of a BBC live-production using virtual technique. The requirements in terms of reliability and robustness were apparent here. Other short examples were given by D. Stricker and M. Op de Beeck on their work in Archeoguide and ATTEST.
The discussions were focussed on usability and benefits of mixed-reality techniques. The most relevant points were:
- One of the keys to successfully deploy MR systems is not seamlessly integrate into existing practices
- 3D and Mixed Reality have a strong potential in terms of enhancing the user experience
- Two important bottlenecks today are tracking and authoring (see AMIRE workshop report)
- For the distribution of Mixed Reality services, it is important to consider the end-2-end quality of service over whatever network and whatever terminal. Not enough attention is given to the central part of the distribution chain.
- Complexity issues should not be underestimated
- Solutions should take into consideration that terminal resources are limited
- It was reminded that simple solutions are often more effective and have a better chance to be implemented
- The possibility of sharing software while protecting IPR was discussed. Proposals to explore various options (eg. documentation, interfaces, converters, contest.) will be made by projects on future occasions
It was agreed that in a few industrial areas the use of mixed-reality has already taken place, but a lot of work has still to be done to make its use wider.
Oliver Grau
BBC
May 20, 2003