Skip to main content

Video Browsing Exploration and Structuring

Objective

Video is a rich source of information. It provides temporal and spatial information about scenes. However, this information is implicitly buried inside the raw video data, and is very inefficiently organized. While the standard sequential frame-based representation of video data is adequate for viewing in a "movie mode", it fails to support rapid access to information of interest that is required in many emerging applications. VIBES proposes new content-based representations of video data, which explicitly emphasize the geometric, photometric, and dynamic components of information. In particular, VIBES will develop methods for rapid video search, hyper-linking, re-animation and view synthesis, which will enhance the content and enrich the experience of video sequences. Video is a rich source of information. It provides temporal and spatial information about scenes. However, this information is implicitly buried inside the raw video data, and is very inefficiently organized. While the standard sequential frame-based representation of video data is adequate for viewing in a "movie mode", it fails to support rapid access to information of interest that is required in many emerging applications. VIBES proposes new content-based representations of video data, which explicitly emphasize the geometric, photometric, and dynamic components of information. In particular, VIBES will develop methods for rapid video search, hyper-linking, re-animation and view synthesis, which will enhance the content and enrich the experience of video sequences.

OBJECTIVES
Video provides continuous coverage of scenes over an extended region both in time and in space. That is what makes it more than a plain collection of images. In VIBES, our objective is to make video a first class data type, which can be searched on content, annotated, hyper-linked, and edited much as text can be now. Furthermore, video has many more modes of information than simple text. For example, it contains scene geometry and extended actions over multiple frames.
Our objectives are also to extract and use these "modes". With these aims, VIBES proposes new ways of exploring and using video that have the potential of leading to significant breakthroughs in video consumption and new industrial, commercial, and home entertainment applications. The tools we develop will enable cut detection, indexing, synthesis, and classification of non-static and non-rigid scenes.

DESCRIPTION OF WORK
The project contains eight interlinked work packages investigating two main themes:
1. Rapid browsing and retrieval: A video or a DVD will be automatically augmented with hyperlinks connecting shots containing a particular actor, type of action, or scene. E.g. all scenes inside the casino in "Casablanca". Such facilities will change the way in which video is addressed, significantly reducing the tedium and inefficiency of current serial video browsing.
2. 3D scene synthesis and human animation models: 3D scene geometry for virtual reality environments will be automatically generated for particular shots. E.g., the yellow brick road in the "Wizard of Oz" could be reconstructed, and a viewer could then walk down it using VR together with virtual actors. 3D dynamical models of actors from classic movies will be learned and used to generate new scenes involving the actors -synthetic thespians- or to replace one actor by another. For example, replace the "hero" in Home Alone or Toy Story by a texture mapped dynamical model of your son or daughter.

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

KUNGLIGA TEKNISKA HOEGSKOLAN
Address
Valhallavaegen 79
100 44 Stockholm
Sweden

Participants (5)

ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE
Switzerland
Address
Ecublens
1015 Lausanne
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
France
Address
Domaine De Voluceau
78153 Le Chesnay
KATHOLIEKE UNIVERSITEIT LEUVEN
Belgium
Address
Oude Markt 13
3000 Leuven
THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF OXFORD
United Kingdom
Address
University Offices, Wellington Square
OX1 2JD Oxford
THE WEIZMANN INSTITUTE OF SCIENCE
Israel
Address
Herzel Street 2
76100 Rehovot