Skip to main content
European Commission logo print header

Flexible and Adaptive Spoken Language and Multi-Modal Interfaces

Objective

We propose a robust and scalable Virtual Personal Assistant within the domain of conversational interaction, incorporating multi-modal input and output in a mobile device with access to an agenda, email and contacts and the potential for rapid integration of other Internet content. The system will be ideal for a 3G mobile network. The development of current and new research is required to generate adequate language recognition models for non-trivial domains and the fusion of multi-modal information with dynamically updated databases (to feed a mobile 3G service). This project deploys new research in natural language understanding, specifically:
(1) Sophisticated dialogue management that adapts to the experience and style of the user; and
(2) An advanced language model capable of integrating and unifying several models of linguistic knowledge.

OBJECTIVES
Mobile phones have now almost saturated the market of major states of the EU; any further commercial advance must therefore come from developments in applications available on the ubiquitous hardware. In this proposal, we shall assemble a robust and scalable Virtual Personal Assistant managing e-mail, calendar and agenda through an intelligent, friendly adaptive multi-modal interaction. The project will use best-of-breed technologies for those modules of a multi-modal system that have matured, and deploy research to develop advanced language understanding and dialogue models. The innovation of FASiL is to pilot a full multi-modal voice portal application that is 3G mobile network ready, along with tools for rapid development of new applications. FASiL targets the languages of UK English, Portuguese and Swedish.

DESCRIPTION OF WORK
FASiL was prepared on an exploratory award grant and responds to key action line VIII.1.11. FASiL addresses thematic priorities of IST (III.3.2) Natural and Multilingual Interactivity; (II.1.3) Mobile and Ubiquitous e-Work and e-Commerce; (V.1.2 CPA 2) Multimodal and Multisensorial dialogue modes.If the quality of life and productivity of EU citizens is to be maintained and improved in an increasingly information dominated society, then people need access to information anywhere and any time. FASiL will provide a Virtual Personal Assistant that addresses this need. It will adapt to the user, learning their preferred mode of interaction. This will result in a more efficient and robust interaction. The issues relating to conversational speech in a multi-modal environment represent a novel set of research issues in modality integration and spoken language processing. FASiL will take high-quality existing modules for the mature functions such as speech recognition and synthesis, and deploys new research for those modules that are not available off-the-shelf, including: (1) sophisticated dialogue management that integrates multi-modal inputs and outputs with e-mail, agenda and contacts databases; and, (2) an advanced language model capable of integrating and unifying several models of linguistic knowledge. Benefits to all and every EU citizen: from its inception, FASiL will assure all citizens will benefit from the platform, including individuals who may be blind of deaf. Best practices to promote accessibility will be followed.

Call for proposal

Data not available

Coordinator

PORTUGAL TELECOM INOVACAO, S.A.
EU contribution
No data
Address
RUA JOSE FERREIRA PINTO BASTO
3810 AVEIRO
Portugal

See on map

Total cost
No data

Participants (7)