Skip to main content

Flexible and Adaptive Spoken Language and Multi-Modal Interfaces

We propose a robust and scalable Virtual Personal Assistant within the domain of conversational interaction, incorporating multi-modal input and output in a mobile device with access to an agenda, email and contacts and the potential for rapid integration of other Internet content. The system will be ideal for a 3G mobile network. The development of current and new research is required to generate adequate language recognition models for non-trivial domains and the fusion of multi-modal information with dynamically updated databases (to feed a mobile 3G service). This project deploys new research in natural language understanding, specifically:
(1) Sophisticated dialogue management that adapts to the experience and style of the user; and
(2) An advanced language model capable of integrating and unifying several models of linguistic knowledge.

OBJECTIVES
Mobile phones have now almost saturated the market of major states of the EU; any further commercial advance must therefore come from developments in applications available on the ubiquitous hardware. In this proposal, we shall assemble a robust and scalable Virtual Personal Assistant managing e-mail, calendar and agenda through an intelligent, friendly adaptive multi-modal interaction. The project will use best-of-breed technologies for those modules of a multi-modal system that have matured, and deploy research to develop advanced language understanding and dialogue models. The innovation of FASiL is to pilot a full multi-modal voice portal application that is 3G mobile network ready, along with tools for rapid development of new applications. FASiL targets the languages of UK English, Portuguese and Swedish.

DESCRIPTION OF WORK
FASiL was prepared on an exploratory award grant and responds to key action line VIII.1.11. FASiL addresses thematic priorities of IST (III.3.2) Natural and Multilingual Interactivity; (II.1.3) Mobile and Ubiquitous e-Work and e-Commerce; (V.1.2 CPA 2) Multimodal and Multisensorial dialogue modes.If the quality of life and productivity of EU citizens is to be maintained and improved in an increasingly information dominated society, then people need access to information anywhere and any time. FASiL will provide a Virtual Personal Assistant that addresses this need. It will adapt to the user, learning their preferred mode of interaction. This will result in a more efficient and robust interaction. The issues relating to conversational speech in a multi-modal environment represent a novel set of research issues in modality integration and spoken language processing. FASiL will take high-quality existing modules for the mature functions such as speech recognition and synthesis, and deploys new research for those modules that are not available off-the-shelf, including: (1) sophisticated dialogue management that integrates multi-modal inputs and outputs with e-mail, agenda and contacts databases; and, (2) an advanced language model capable of integrating and unifying several models of linguistic knowledge. Benefits to all and every EU citizen: from its inception, FASiL will assure all citizens will benefit from the platform, including individuals who may be blind of deaf. Best practices to promote accessibility will be followed.

CSC - Cost-sharing contracts

PORTUGAL TELECOM INOVACAO, S.A.
Rua Jose Ferreira Pinto Basto
3810 Aveiro
Portugal

(7)

CAPGEMINI SVERIGE AB
Sweden

161 24 Bromma
MEDIALABEUROPE LIMITED
Ireland
First Floor Fitzwilton House Wilton Place
Dublin 2
ROYAL NATIONAL INSTITUTE OF THE BLIND
United Kingdom
105 Judd Street
WC1H 9NE London
SCANSOFT AACHEN GMBH
Germany
Kackertstrasse 10
52072 Aachen
THE ROYAL NATIONAL INSTITUTE FOR DEAF PEOPLE
United Kingdom
19-23 Featherstone Street
EC1Y 8SL London
THE UNIVERSITY OF SHEFFIELD
United Kingdom
Firth Court, Western Bank
S10 2TN Sheffield
VOX GENERATION LIMITED
United Kingdom
90 Fetter Lane
EC4A 1JP London