Skip to main content
EnglishEN
European Commission logo print header

Unified Intelligent Access to Heterogeneous Audiovisual Content

Objective

The overall objective of FAETHON project is to develop an integrated information system offering enhanced search and retrieval capabilities to users of audiovisual (a/v) archives. This novel system will exploit the advances in handling a/v content and metadata, as introduced by MPEG-4 and MPEG-7, to provide access characterized by semantic phrasing of the request (query), unified handling and personalized response. This will be achieved by developing algorithms and software for (i) extracting high level semantic out of syntactic and low level semantic information contained in the a/v archives, (ii) filtering the responses of the latter on the basis of continuously updated profiles of individual users. To this end, state of the art technologies will be used and new algorithms in the fields of fuzzy and hybrid systems will be developed. Novel database schemes for multidimensional indexing will be employed. The overall objective of FAETHON project is to develop an integrated information system offering enhanced search and retrieval capabilities to users of audiovisual (a/v) archives. This novel system will exploit the advances in handling a/v content and metadata, as introduced by MPEG-4 and MPEG-7, to provide access characterized by semantic phrasing of the request (query), unified handling and personalized response. This will be achieved by developing algorithms and software for (i) extracting high level semantic out of syntactic and low level semantic information contained in the a/v archives, (ii) filtering the responses of the latter on the basis of continuously updated profiles of individual users. To this end, state of the art technologies will be used and new algorithms in the fields of fuzzy and hybrid systems will be developed. Novel database schemes for multidimensional indexing will be employed.

OBJECTIVES
It becomes clear among the research community dealing with content-based a/v data retrieval and new emerging related standards, such as the MPEG-7, that the results to be obtained will be ineffective, unless major focus is given to the semantic information level, defining what most users desire to retrieve. Mapping, however, low level, subsymbolic descriptors of a/v archives to high level symbolic ones is in general difficult, even impossible with the current state of technology. It can, however, be tackled when dealing with specific application domains. The project is targeting the extraction of semantic information from a/v and text related data taking into account: a) The nature of useful queries that users may issue. This is only a portion of the general set of questions related to 'content understanding'. Using all types of multimedia information of the archives makes the task more tractable. b)The context determined by user profile.

DESCRIPTION OF WORK
FAETHON will create a novel system exploiting the advances in handling audiovisual (a/v) content and related metadata, as introduced by MPEG-4 and MPEG-7, to offer advanced access services characterized by the tri-fold 'semantic phrasing of the request (query)', 'unified handling' and 'personalized response'. The proposed system will play the role of an intermediate access server residing between end users and multiple heterogeneous archives organized according to new MPEG standards. The core technological target of the project is to blend the achievements in characterizing a/v content - especially visual and acoustical - with innovative hybrid intelligence technologies in order to: (i) offer unified semantic views to existing a/v archives, beyond the classification schemes and subject indexes of each archive, (ii) personalize those views according to the context defined by the profile of individual users. The project will produce novel tools and methods for extracting high-level semantic information through Dynamic Thematic Categorization (DTC) of a/v content units and Detection of Events and Composite Objects (DECO) within the archived a/v material. DTC will provide a fuzzy association of a/v units to the nodes of a Thematic Categorization Structure reflecting the interests of specific users/groups. DECO corresponds to the identification of composite objects or events that have semantic interpretation. A variety of associations will be possible at this level. The project will reduce their number by exploring the specific user behavior and constructing "personalized" sub-associations; consequently, it will adapt its performance to the specific users? interests, thus increasing its information retrieval efficiency. Using statistics and relevance feedback will be examined to assist personalization. Hybrid intelligent techniques, mainly neurofuzzy approaches, will be explored for constructing/learning the most appropriate semantic associations.
FAETHON has accomplished the original research objectives, as foreseen in the Workplan, producing all expected deliverables and associated work. Moreover, the FAETHON partners have moved further in many research directions, especially in the semantic analysis issue. In particular, since the beginning of the project it has become clear that semantic unification can only be achieved if based on appropriate handling of existing a-priori knowledge. A basic component of the developed system is its knowledge part, composed of the Encyclopaedia, the Thesaurus and the Index. MPEG-7 XML based descriptions and representations are the means to achieve unification of access to archives' content. Novel fuzzy technologies have been developed for assessing and handling this content.

The generated FAETHON System consists of the following:
1. The Database Management subsystem, including three main modules (data structures): Encyclopaedia: The Faethon encyclopaedia consists of a set of description schemes for the definition of all semantic entities and their relations. It actually contains all knowledge of semantic information used in the Faethon system, being coded in three types of information: semantic entities, semantic relations and a Thesaurus. Index: The index is used by the Faethon search engine for fast retrieval of documents related to specific semantic entities. It contains sets of document locators in the encyclopaedia. User profiles: User profiles contain the usage history and user preferences and authentication and authorization related information;
2. The Searching subsystem, including: Query Analysis Module: It takes advantage of the semantic entity relations in the encyclopaedia and the Faethon thesaurus to expand queries. It consists of three operations: query interpretation, expansion and personalization. The final result is a structure of semantic entities with confidence values. Search Engine: It takes as input a semantic expression and the metadata expression of the user query and produces as output a Faethon query. It consists of two operations: semantic transformation and query formulation;
3. The Semantic Unification subsystem, including: Encyclopaedia Update: It updates the thesaurus that associates semantic entities through relations, as well as the semantic entities. It consists of two operations: semantic entities and relations update and fuzzy thesaurus update. The first is based on application of the DTC and DECO modules to new MPEG-7 multimedia data, with a human expert correcting the results. Then, the fuzzy thesaurus is automatically updated. Dynamic Thematic Categorisation: The DTC performs matching between the textual metadata associated to the archived material and the entities of the encyclopaedia. It takes as input the thematic categories stored in the encyclopaedia, unifies them with the thematic categories of the a/v archives and stores (as links) the a/v units that belong to each thematic category. The DTC output is stored in the index, for fast document retrieval. Detection of Events and Composite Objects: The DECO module performs detection of specific composite objects or events that have semantic interpretation. It takes as input the semantic entities stored in the encyclopaedia and stores (as links) the a/v units that contain each semantic entity and complicated semantic structure, with a weight representing the respected degree of confidence. Its output is stored in the index;
4. The Personalisation subsystem, including: User Profile Update: It includes the usage history that is updated after the end of a user query. It then extracts the user preferences with the aid of the encyclopaedia and the a/v unit descriptions from the usage history. A/V Classification: It performs ranking of the retrieved documents from the archive based on user interests contained within the preferences of the user profiles. Presentation Filtering: It receives as input the output of the a/v classification module. It further updates the ranking of documents based on other components of the user preferences. Its output, the Faethon response, is sent to the communication module of the user interface;
5. The User Interface subsystem, including the User Interaction, User Presentation and User Communication Modules;
6. The Individual a/v Archive Interface subsystem, including the Query Translation, Response Assembly and Archive Profile Modules.

Fields of science

Programme(s)

Topic(s)

Call for proposal

Data not available

Funding Scheme

CSC - Cost-sharing contracts

Coordinator

ALTEC INFORMATION AND COMMUNICATION SYSTEMS S.A.
EU contribution
No data
Address
PATMOU 12
15123 MAROUSSI - ATHENS
Greece

See on map
Total cost
No data

Participants (7)

Share this page

Download

Last update: 13 June 2005

My Booklet

Permalink: https://cordis.europa.eu/project/id/IST-1999-20502

European Union, 2024