Skip to main content

Audio visual indexing and retrieval for non it expert users

Article Category

Article available in the folowing languages:

Retrieval of television archived material using intelligent descriptors

The vast amount of programmes created in the short but very fruitful history of television has never been easily retrieved simply because there were no tools for indexing and retrieval of archived material. A European funded project has succeeded in developing innovative descriptors for TV programmes in order to facilitate audio-visual content analysis and indexing. These innovative descriptors make it possible to identify footage, such as scenes from a football match, consequently opening up a new horizon of business opportunities for the broadcast services.

Digital Economy

Audio Visual Indexing and Retrieval for non-IT expert users, AVIR, is a transnational European project, partly funded by the European Commission. The project through the development of content descriptors of television programmes has created innovative ways for searching and browsing through the programmes. Key-frame browsing, news search, similarity retrieval and many other search functions are now available. The results of the project are protected by several patents and have been thoroughly tested and already demonstrated in exhibitions in the area of broadcasting and multimedia. AVIR is based on novel algorithms and new routines that were developed for the extraction of content descriptors and their subsequent addition on existing metadata. Basic information can be extracted from either or both the audio or video channel. Even low-level information such as camera motion parameter estimation, micro-segment based or shot based mosaic extraction, speech recognition and even dominant colour extraction and script warping can be utilized. Such information, though considered semantically low, is also extremely useful since firstly it can be used on its own for easy browsing and searching and secondly it can be further processed to provide more semantically meaningful information to the user. The descriptors thus produced are based on the MPEG-7 format to which the AVIR project actively contributed, having a central role in its AHG and XM developments. The descriptors are organized according to schemes developed by the AVIR project. Project results have been integrated into a single system for metadata production. The system called the AVIR Service Provider System is a distributed, layered architecture incorporating the audio and video analysis routines as functional modules. The provider naturally includes components for MPEG-7 compliant metadata import, storage, management and export. The Service Provider System (SPS) is integrated into an already existing commercial product developed by one of the project partners. The metadata reaches the final destination, i.e. the consumer, via a server that delivers both data and content to the consumer's terminal via a DVB channel. This server was also developed within the AVIR project and the transmission system has been successfully tested and demonstrated. The advanced retrieval features offered by the AVIR consumer system, along with storage capabilities and an intuitive, multimodal graphical user interface has strong business potential. The results of the AVIR project, being at the leading edge of technology, have already been published and project partners are seeking multimedia industries and broadcast services for immediate exploitation of their developments on a joint venture agreement basis.

Discover other articles in the same domain of application