Objective
The objective of THISL is to show the feasibility of integrating state of the art Natural Language Processing (NLP) and Large Vocabulary Continuous Speech Recognition (LVCSR) technologies, towards advanced multimedia applications. In this framework, the present proposal will focus on R&D aimed at retrieving multimedia information (written or spoken text) using a spoken language interface.
The industrial relevance of such an application is apparent and was made very clear by the huge interest raised by the Carnegie Mellon University (CMU) Informedia realtime "news-on-demand" demonstration at the recent ARPA Spoken Language Technology meeting (Feb 1996, Arden House NY). The indexing and retrieval of television, radio and text news via a spoken language interface is a particularly compelling application which should be demonstrated at the end of this project. This technology would have potential applications with the advent of 500-channels and interactive TV. It could offer real-time, content-based access to ongoing programmes through their soundtrack, or sound-based navigation in the programme network.
The expected result of the project is a real-time prototype system for navigating in the soundtrack of a TV news broadcast. Significant intermediate results will include transcription of broadcast speech, development of audio editing tools, content-based retrieval from audio/video archives and a robust spoken language interface for search and retrieval of multimedia data.
The general approach will involve the integration of the LVCSR technology developed in the ESPRIT WERNICKE Basic Research project and the NLP technology developed by Thomson for spoken query understanding. A substantial output of the WERNICKE project was a flexible, efficient and accurate LVCSR system (able to handle open lexica) achieved through a new technology combining hidden Markov models (HMMs) and artificial neural networks (ANNs).
In terms of research, the current project will also directly benefit from the ongoing LTR SPRACH (20077) project aiming at further improving HMM/ANN-based LVCSR systems. As a consequence, most of the remaining work will focus on problems arising from the application domain and the integration of speech and language, particularly: (1) increasing robustness of the speech recognise on multimedia and broadcast speech (e.g. presence of music, mix of studio and telephone speech), (2) phrase and keyword spotting, and (3) the robust processing of spoken queries aimed at soundtrack transcripts.
The consortium has six proposers, with a clear interest in that application but with complementary areas of expertise. Sheffield University (SU), UK, has a large expertise in large vocabulary speech recognition, with particular emphasis on language modelling. The British Broadcasting Corporation (BBC), UK, has a great experience and interest in audio and video signal processing. As well as having access to large speech files, the BBC has considerable expertise in the editing and indexing of broadcast material, and an understanding of what is required and might be beneficial for the broadcast industry. Their main interests in this project are: (1) automatic indexing of broadcast news, and (2) automatic retrieval of video data by locating keywords in the audio track.
The Faculté Polytechnique de Mons (FPMs), B, has a large expertise in speech recognition, with particular emphasis on hybrid HMM/ANN speech recognition systems and keyword spotting. SoftSound, UK, is a start-up company interested in the integration of the different technologies and the development of a real-time demonstration system. Thomson-CSF/LCR, F, has a large experience in human-computer dialogue, speech understanding and natural language processing. IDIAP, CH, has considerable experience in speech and speaker recognition; its primary role will be to adapt the techniques to French speech. THISL will also benefit from further collaboration with their WERNICKE and SPRACH subcontractor ICSI (Berkeley CA, USA), which has been shown particularly successful in terms of technical input, as well as advanced software and hardware tools.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- natural sciences computer and information sciences software
- engineering and technology electrical engineering, electronic engineering, information engineering electronic engineering signal processing
- natural sciences computer and information sciences data science natural language processing
- natural sciences computer and information sciences artificial intelligence computational intelligence
You need to log in or register to use this function
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Data not available
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Coordinator
S10 2OH Sheffield
United Kingdom
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.