CORDIS
EU research results

CORDIS

English EN
Flexible and Adaptive Spoken Language and Multi-Modal Interfaces

Flexible and Adaptive Spoken Language and Multi-Modal Interfaces

Objective

We propose a robust and scalable Virtual Personal Assistant within the domain of conversational interaction, incorporating multi-modal input and output in a mobile device with access to an agenda, email and contacts and the potential for rapid integration of other Internet content. The system will be ideal for a 3G mobile network. The development of current and new research is required to generate adequate language recognition models for non-trivial domains and the fusion of multi-modal information with dynamically updated databases (to feed a mobile 3G service). This project deploys new research in natural language understanding, specifically:
(1) Sophisticated dialogue management that adapts to the experience and style of the user; and
(2) An advanced language model capable of integrating and unifying several models of linguistic knowledge.

OBJECTIVES
Mobile phones have now almost saturated the market of major states of the EU; any further commercial advance must therefore come from developments in applications available on the ubiquitous hardware. In this proposal, we shall assemble a robust and scalable Virtual Personal Assistant managing e-mail, calendar and agenda through an intelligent, friendly adaptive multi-modal interaction. The project will use best-of-breed technologies for those modules of a multi-modal system that have matured, and deploy research to develop advanced language understanding and dialogue models. The innovation of FASiL is to pilot a full multi-modal voice portal application that is 3G mobile network ready, along with tools for rapid development of new applications. FASiL targets the languages of UK English, Portuguese and Swedish.

DESCRIPTION OF WORK
FASiL was prepared on an exploratory award grant and responds to key action line VIII.1.11. FASiL addresses thematic priorities of IST (III.3.2) Natural and Multilingual Interactivity; (II.1.3) Mobile and Ubiquitous e-Work and e-Commerce; (V.1.2 CPA 2) Multimodal and Multisensorial dialogue modes.If the quality of life and productivity of EU citizens is to be maintained and improved in an increasingly information dominated society, then people need access to information anywhere and any time. FASiL will provide a Virtual Personal Assistant that addresses this need. It will adapt to the user, learning their preferred mode of interaction. This will result in a more efficient and robust interaction. The issues relating to conversational speech in a multi-modal environment represent a novel set of research issues in modality integration and spoken language processing. FASiL will take high-quality existing modules for the mature functions such as speech recognition and synthesis, and deploys new research for those modules that are not available off-the-shelf, including: (1) sophisticated dialogue management that integrates multi-modal inputs and outputs with e-mail, agenda and contacts databases; and, (2) an advanced language model capable of integrating and unifying several models of linguistic knowledge. Benefits to all and every EU citizen: from its inception, FASiL will assure all citizens will benefit from the platform, including individuals who may be blind of deaf. Best practices to promote accessibility will be followed.

Leaflet | Map data © OpenStreetMap contributors, Credit: EC-GISCO, © EuroGeographics for the administrative boundaries

Coordinator

PORTUGAL TELECOM INOVACAO, S.A.

Address

Rua Jose Ferreira Pinto Basto
3810 Aveiro

Portugal

Administrative Contact

Nuno BEIRES

Participants (7)

Sort alphabetically

Expand all

CAPGEMINI SVERIGE AB

Sweden

MEDIALABEUROPE LIMITED

Ireland

ROYAL NATIONAL INSTITUTE OF THE BLIND

United Kingdom

SCANSOFT AACHEN GMBH

Germany

THE ROYAL NATIONAL INSTITUTE FOR DEAF PEOPLE

United Kingdom

THE UNIVERSITY OF SHEFFIELD

United Kingdom

VOX GENERATION LIMITED

United Kingdom

Project information

Grant agreement ID: IST-2001-38685

  • Start date

    1 October 2002

  • End date

    30 September 2004

Funded under:

FP5-IST

  • Overall budget:

    € 6 407 258

  • EU contribution

    € 3 472 261

Coordinated by:

PORTUGAL TELECOM INOVACAO, S.A.

Portugal