Skip to main content
European Commission logo print header

Interactive semantic video search with a large thesaurus of machine-learned audio-visual concepts

Project description

Search Engines

Video is vital to society and economy, playing a key role in news, cultural heritage documentaries, and surveillance and is becoming the natural form of communication for the Internet and mobile phones. Consumers and professionals alike need advanced storage and search technology for the management of large-scale video assets. This project takes on the challenge of creating a substantially enhanced semantic access to video, implemented in a search engine.

Video search engines result from the progress in many technologies: visual analysis; audio processing; machine learning; visualization, and interaction technology. A good measure of the overall progress is the TRECVID competition, posing increasingly hard and realistic problems with the obligation to discuss methods openly. In 2005, the best interactive video search came from the coordinating partner of this project.

This project will boost the performance of video search by basing it on a 10-fold enhancement of the thesaurus characterizing audio, visual or mixed-media content. In the US, a similar grand challenge has been posed. This project's approach is to let the system learn many, possibly weaker, detectors instead of modelling a few of them carefully. The combination of many detectors describing different aspects of the video content will render a much richer basis for the semantics.

The Consortium presents excellent expertise and resources: the machine learning is lead by Univ. of Surrey; video stream processing is lead by CERTH, Greece. The component audio event detection, is lead by INESC-ID, Portugal; visual image processing is lead by Univ. of Amsterdam; The Univ. of Florence leads the efforts in interaction, and CVC, Spain leads software consolidation. Finally, Beeld & Geluid, the Netherlands, and FRD, Italy, as application stakeholders, provide data and perform evaluation and dissemination.

Call for proposal

See other projects for this call



Kruislaan 403
1098 SJ Amsterdam

See on map

Activity type
Higher or Secondary Education Establishments
EU contribution
No data

Participants (7)