CORDIS - EU research results

Effective Multilingual Interaction in Mobile Environments

Project description

Cognitive Systems, Interaction, Robotics

The EMIME project will help to overcome the language barrier by developing a mobile device that performs personalized speech-to-speech translation, such that the a user's spoken input in one language is used to produce spoken output in another language, while continuing to sound like the user's voice. Personalisation of systems for cross-lingual spoken communication is an important, but little explored, topic. It is essential for providing more natural interaction and making the computing device a less obtrusive element when assisting human-human interactions. We will build on recent developments in speech synthesis using hidden Markov models, which is the same technology used for automatic speech recognition. Using a common statistical modelling framework for automatic speech recognition and speech synthesis will enable the use of common techniques for adaptation and multilinguality. Significant progress will be made towards a unified approach for speech recognition and speech synthesis: this is a very powerful concept, and will open up many new areas of research. In this project, we will explore the use of speaker adaptation across languages so that, by performing automatic speech recognition, we can learn the characteristics of an individual speaker, and then use those characteristics when producing output speech in another language.
Our objectives are to: Personalise speech processing systems by learning individual characteristics of a user's speech and reproducing them in synthesised speech; Introduce a cross-lingual capability such that personal characteristics can be reproduced in a second language not spoken by the user; Develop and better understand the mathematical and theoretical relationship between speech recognition and synthesis; Eliminate the need for human intervention in the process of cross-lingual personalisation; Evaluate our research against state-of-the art techniques and in a practical mobile application.

Call for proposal

See other projects for this call


EU contribution
€ 700 544,00
EH8 9YL Edinburgh
United Kingdom

See on map

Scotland Eastern Scotland Edinburgh
Activity type
Higher or Secondary Education Establishments
Administrative Contact
Angela Noble (Miss)
Total cost
No data

Participants (6)