Service Communautaire d'Information sur la Recherche et le Développement - CORDIS


SMADA Résumé de rapport

Project ID: IST-1999-10667
Financé au titre de: FP5-IST
Pays: Netherlands

Speech Driven Multi-modal Automatic Directory Assistance (SMADA): Human factors experiments in the Netherlands

The SMADA project aimed at an improved functionality and usability of automated services that use automatic speech recognition (ASR) in their user interface, either as the only input/output modality (i.e. over the telephone) or as one of the modalities in multi-modal interfaces. The results of the Human Factors experiments in the Netherlands are described below.

The HF experiments carried out by Katholieke Universiteit Nijmegen (KUN) focused on the usability of speech driven multimodal interaction. KPN and KUN experimented with a combination of speech and pen input in applications where users must complete a form that requires textual input, for example for names. If the list of possible names is very large, browsing through a list is not attractive. Instead, one would like to be able to enter a name directly, either via speech or via some kind of soft keyboard if a full-fledged keyboard is not available.

The most important results of the Human Factors experiments conducted by KPN and KUN can be summarised as follows:

- Multimodal applications are best designed from scratch. Deriving these applications from existing graphics or speech-only services tends to result in sub-optimal designs.

- ASR systems used in multimodal form filling applications must have specific functionalities. It is especially important that the lexicon and language models can be adapted on-line, to optimally cater for the input that is expected for a specific field in the form.

- Uniformed users do not spontaneously understand the way in which they can combine speech and pen input. For multimodal services to develop rapidly a large degree of standardisation of the interfaces for such services is highly desirable.

- Users prefer speech input over alternative input mechanisms if a full fledged keyboard is not available. However, in case of persistent ASR errors they learn how to use alternative input methods to advantage.

- Allowing users to select the first letter of a name through a soft keyboard and subsequent re-processing the last spoken input utterance, appears to be a very powerful method for dealing with ASR errors.

More information on the SMADA project can be found on the project’s website:

Informations connexes


Louis BOVES, (Head of Department)
Tél.: +31-24-3612902
Fax: +31-24-3612907