Objective
The usefulness of archived audiovisual material is strongly dependent on the quality of the accompanying annotation. Currently this is a labour-intensive process, which is therefore limited in the amount of detail that can be stored. In particular, in real-time applications (such as live broadcast events) it is unrealistic to add much manual annotation. The proposed information management system will automatically extract descriptive features, using MPEG-7 descriptors where relevant, and associate these features with a small thesaurus relevant to the subject matter. In this project, the subject matter will be limited to sports events. The features will include video, text, speech and other audio features. These features will be associated with the thesaurus by means of a training process. In this way the user will be able to make text-based queries on the audiovisual material, using only the automatically-extracted annotation.
Objectives:
The aim of the project is to develop techniques for automatic segmentation and semantic annotation of sports videos. Such material typically originates "live", thus making detailed manual annotation impractical. The level of annotation should be sufficient to enable simple text-based queries. The target will be to segment the material into shots, and to group and classify the shots into semantic categories (type of sport). To do this, the system will extract information from each shot, based on speech and text recognition, and identify the highlights from the audio track and from visual audience reactions. A training system will then be developed that will then associate these features with a small thesaurus relevant to the subject matter.
Work description:
The project is based on the construction of several software modules that extract various different modes of information (speech, other audio cues, encapsulated text, other video cues). Each module functions fairly independently of the others, so that the development of the modules can proceed in parallel. The video module can be further roughly subdivided into shot detection, spatiotemporal object extraction and mosaicing, and object characterisation and recognition. The output of these modules is fed into a contextual annotation module, which is able to associate the automatically-extracted features with an internal lexicon. The output is a text-based summary of the video material. The associative mechanism is trained for the specific application - in this case sports material. A user interface will be developed which will enable the user to browse through the video database, using the automatically-extracted text summary to link to the original video material. Special attention will be paid to the flexibility and user-friendliness of this interface. As the software components are integrated into a complete system, attention will be paid to the development of testing methodology, and to an evaluation of the prototype. The direction of this work will be guided strongly by input from the user partners. Recent work on video annotation has highlighted the difficulty of locating sufficient material with adequate ground truth information. Thus the compilation, cataloguing and assessment of suitable test material forms an important component of this part of the work.
Milestones:
M1 (at 9 months): User requirements and system specification completed; Graphical User Interface able to operate on manually-generated annotation.
M2 (at 18 months): Audio and video segmentation, shape analysis and motion characterisation complete.
M3 (at 24 months): Skeleton system completed with trial versions of all workpackage modules
At project completion, we expect:
(1) to have a working prototype of the system, and
(2) to have made contributions to standardisation bodies.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- natural sciences computer and information sciences software
- natural sciences computer and information sciences databases
You need to log in or register to use this function
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Data not available
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Coordinator
KT13 0XW WEYBRIDGE
United Kingdom
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.