Objective
Goal:
To develop multilingual technology for a speech-derived synthetic face that gives essential visual speech information to hearing impaired users of telephone and other voice channels. This group is significantly handicapped in access to spoken information. Visible face movements are an effective means of enhancing speech intelligibility especially for hearing-impaired persons, and for all in noise. These developments will be assisted by the rapid development of multi-modal speech technology and PC processing power. Background from KTH in this area is unique in Europe. Use will be made of existing multilingual speech databases. Outcomes will include a market definition, a specification of visual information delivery and speech recognition requirements, a complete multilingual prototype, user-based trials of the prototype and a final market assessment based on these trials and a review of implementation in the context of European Telecoms.
DESCRIPTION OF WORK
The project will be based on background technology from the KTH multi-modal speech group, whose achievements in this area are unique in Europe, and existing multilingual speech databases.
The KTH technology comprises two main components.
1/ a system for generating a computer image of a talking 3D head, currently controlled from Swedish phoneme strings
2/ a phoneme level speech recogniser already developed to provide control data of proven utility for the talking head.
Project tasks:
1. Commercial Evaluation (Telia, RNID)
1a. Initial review of market for this technology in relation to such alternatives as videophone, computer-based video links, text displays driven by speech recognition
1b. Final market assessment drawing on outcomes of multi-lingual trials of project prototype (see 4.)
2. Research to specify speech recognition requirements in terms of quality of visual face information (UCL, KTH, IvD, RNID)
2a. definition of speech (viseme) features that need to be accurately recognised
2b. assessment of user tolerance of class and frequency of recognition error
2c. review of existing knowledge of tolerable audio-visual delay and audio-visual asynchrony
3. Development of speech recognition software for a complete demonstrator for the target languages, Dutch, English and Swedish, including assessment of language-independence of recogniser performance and need for language and speaker specific adaptation (KTH, UCL, IvD)
4. Assembly (KTH, Telia) and multilingual testing of full prototype designed for home and workplace use (all partners).
MILESTONES
1. Preliminary definition of user requirements and likely market in Europe - feeding into system design
2. Specification of visual information requirements to determine target performance for speech recognition
3. Speech recognition system optimised for this application & spin-offs to other face image generation applications.
4. Multilingual tested prototype for generation of talking head from live telephone speech.
5. Final full market assessment based on user-evaluations of the prototype.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- natural sciences computer and information sciences software
- natural sciences computer and information sciences databases
You need to log in or register to use this function
We are sorry... an unexpected error occurred during execution.
You need to be authenticated. Your session might have expired.
Thank you for your feedback. You will soon receive an email to confirm the submission. If you have selected to be notified about the reporting status, you will also be contacted when the reporting status will change.
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Data not available
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Coordinator
100 44 STOCKHOLM
Sweden
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.