The scientific goal of this assessment project is to develop and test a low bit-rate high quality video compression that is based on new innovative segmentation algorithms. These segmentation algorithms will enable to extract and track the objects among consecutive video frames. The plan is to setup framework of tools for multimedia applications that are content-based. An automatic interaction with the contents to achieve a low bit-rate video compression will be based on the extracted segments. This will enable substantial improvements of the coding efficiency and will enable also to have efficient coding of multiple concurrent data streams. The proposed video codec will not necessitate extensive computing power.
The objectives of this project are to develop and test a new innovative framework to extract visual objects that will enable to achieve a very low bit rate of high quality video compression that will outperform existing methods. It can also be added later to MPEG-4 standard, which recommends to utilize object manipulation to achieve low bit rate compression. But how to extract these visual objects remains an open problem for MPEG-4. The proposed methodology can substantially enhance the performance of MPEG-4 and MPEG-7.
Typical applications for the proposed technology are:
- interpersonal real-time video communication with low bit rate;
- broadcast video distribution with low bit rate;
- content manipulation of video content for home video production;
- mobile multimedia;
- content-based storage and retrieval;
- streaming video on the Internet/Intranet;
- digital set-top box and many more.
DESCRIPTION OF WORK
The low bit rate high quality compression will be based on a sequentially combining baseline encoder, segmentation of the motion vectors, motion estimation and compensation, extraction of the moving segments from the motion vectors by segmentation of the projected 3-D space into 1-D space, tracking them and coding each segment with two bytes, The residual frame (the frame without the moving objects) will be compressed by our adaptive wavelet based video codec or by DCT based codec. The following steps of the algorithm will be necessary to achieve high quality low bit rate.
Application of motion compensation and identification of:
1. the motion vectors;
2. Segmentation of the motion vectors to produce the segments. Each segment necessitates 2 bytes to be losslessly preserved;
3. Each frame is broken according to the contents: moving segments and the residual frame with no objects;
4. The residual frame (without the segments) is compressed using our fast still wavelet compression that is based on optimal bit allocation or by DCT based codec (as in MPEG-4);
5. An optimal bit allocation for differentiating between frames in each group and their predictions;
6. When we have the segments of the central frame and their motion parameters, we predict from them the other frames in the group.
By identifying differently moving objects that are then represented by a few motion parameters, great coding gains may be achieved. We can enhance the above method using a split-and-merge approach to segment the scene into regions of homogeneous motion parameters. Then the 3D motion parameters are estimated for each region. A quad tree structure provides an efficient way to represent the object boundaries, as well as the segmentation mapping. The objects can be of any shape and topology. A special careful analysis of the effect of the algorithm proposed algorithm on the segmentation of the motion vectors will enable us to find the correct boundaries of the segment sinc e motion vectors can lead us to a good match not necessarily at the boundaries of the whole segments.
Funding SchemeACM - Preparatory, accompanying and support measures
100 44 Stockholm