Objective
POP-EYE will combine subtitle recognition, natural language indexing and partial translation to develop a film and video indexing tool. These multilingual indexes can then be used by broadcasting companies to help producers locate, retrieve and reuse video fragments. This will open up new markets for customers by providing better, cheaper and faster access to video material. The availability of subtitles comes from the provision of film and video material for viewers with hearing difficulties and from the use of subtitling, rather than dubbing, to present foreign language video material in some countries. POP-EYE will extract complete scripts from the subtitles using format conversion. It will then build multilingual indexes to these scripts using full text indexing, phrase indexing and partial translation.
Indexing and retrieval of multimedia material is time consuming and expensive. Time consuming because much of it has to be done manually, and expensive because it has to be done by specialist experts. There are no systems that can watch and interpret moving images, nor systems that can listen to and understand spoken text. Tools are needed to help TV producers find video and film fragments for new productions, such as documentaries and training programmes.
Information systems for archiving multimedia have been taken up slowly by industry. The main reason is that they do not automate the most expensive and time consuming activity which is profiling. Profiling involves attributing features to multimedia objects so that they can be successfully retrieved. These features cover 'non-contents' descriptions which include the author, date, publisher and addressee, 'contents' descriptions which include index keywords and class codes, and judgements which concern its usefulness. In most situations this takes up more than 75% of the total costs of archiving.
Modern multimedia and document information systems contain not only structured information about the documents, but also the documents themselves, in their various formats such as text, images, sound and video. Users can not only look for a document, they can look at it whether it be still or moving pictures, or continuous such as sound and video sequences.
Modern systems cannot perform fully automatic document profiling at an acceptable level of quality. They generally lack the ability to allow the user to locate suitable material despite the rapid proliferation of systems which can store and distribute multimedia documents such as the World Wide Web. Research and systems sales figures both show that qualitatively acceptable automation of document profiling requires more than just straight forward application of current engineering techniques as in text indexing systems. This especially applies to film and video archiving and retrieval which suffers from a lack of suitable tools and infrastructure.
Pop-Eye will make of use subtitles linked to video and film sequences, in a completely new method for indexing this material. One of the main sources of subtitled video material will be sequences that have already been subtitled for deaf viewers. Other sources include foreign language film and video material. Subtitle recognition, natural language indexing and partial translation will be used to extract complete scripts from these subtitles to build multi-lingual indexes which will be used by the retrieval tool to find the required sequences.
The main goal of Pop-Eye is to develop a film and video archiving and retrieval tool that will provide access to large libraries of continuous information objects (video) without the current large resource requirements in terms of network traffic and search time. This is to be developed for retrospective archiving - the conversion and indexing of all material that is in the current archive - and adaptation of the daily archiving process.
The co-ordinating partner and three others are large broadcasting companies, each having its own particular expertise with subtitling in Dutch, English and German. Two other partners are universities which have long term experience in linguistics and language engineering, combined with expertise in multimedia processing and telematics. These are backed up by two research institutes who are well known for their success in developing and selling natural language technology that goes far beyond prototypes, and a software house with long term experience in developing for television companies.
The demonstrator will be prepared with a database using video material supplied by the project partners and will be used to test the system. A stripped down version for use over the Internet will also be made available to test its retrieval performance. Parts of it will be processed for publication on CD-ROM, to test the functionality of supplying indexes to sets of video material held in large libraries.
The project co-ordinator will use the POP-EYE technology to build their own multimedia archive. They are currently using an external archive which is costly and inefficient. The other three broadcasting partners will use the technology to automate their archives and to enhance the present search mechanisms so that the required video sequences can be accessed more efficiently. The aim is also to reduce the costs of profiling videos; to improve precision, reduce noise and improve user- friendliness.
To build the demonstrator, the consortium will use profiling techniques based on natural language, multimedia and dialogue processing and fuzzy matching.
Natural language processing: Indexes will be built consisting of phrases, in more than one language where texts are available. Machine translation will be used to convert these phrase index entries into one common database language, which will enable it to point to video sequences subtitled in different languages.
Multimedia processing: The index to video sequences will be built from the scripts extracted from the programme subtitles. These entries will be stored with digitised video fragments together with the text and pointers to the video clips, to build a multimedia database.
Dialogue processing: Scripts extracted from the subtitles will often consist of spoken dialogue which often does not give any insight into the kind of video fragments they accompany. To improve the indexing quality, the dialogue will be filtered using a robust dialogue model combining dialogue theory, heuristics and statistical processing.
Fuzzy matching: With natural language indexing it is more difficult to match a query with a complete term in the index than when querying with single words. Both the query and the index will be broken down into sets of morphological primitives which can then be matched by intersection.
The main user group is made up of the current professional users: the programme makers and staff of the film and video archives of the broadcasting partners. Other user groups outside these companies are other broadcasting and advertising companies, film archiving organisations, and other organisations that also need access: libraries, government organisations, newspaper companies, press agencies, and citizens.
Professional users will benefit from better and much cheaper access to video material, including foreign material. They will be able to search and locate the relevant sequences while sitting at their desk, without the network traffic involved in downloading video material. This will enable them to make better quality, cheaper productions. Broadcasting companies will also be able to more easily shape foreign language spoken TV productions to their own needs.
Small video producers will be able to make their material available on the Web, or to market it via CD-ROM, thus securing a wider audience and increased economic viability. The cost effectiveness of subtitling for deaf viewers will be increased allowing a wider range and quantity of material to be made available to them.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- humanities languages and literature general language studies
- natural sciences computer and information sciences databases
- natural sciences computer and information sciences data science natural language processing
- social sciences sociology industrial relations automation
- natural sciences computer and information sciences internet world wide web
You need to log in or register to use this function
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Data not available
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Data not available
Coordinator
1202 LL Hilversum
Netherlands
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.