Skip to main content
European Commission logo print header
Contenu archivé le 2024-04-16

A Multi-Modal Interface for Human-Machine Interaction with Knowledge-Based Systems

Objectif

The MMI2 project undertakes research into multi-modal interfaces to knowledge-based systems (KBS).
The project is developing an advanced human computer interface to support cooperative dialogue between users and knowledge based systems. The interface supports natural language, command language, graphical display, direct manipulation and gesture. Several tools are incorporated to allow the graphical visualization of the state of the knowledge base, its suggested designs for local area computer networks, and interactive charts and graphs. The mechanism for designing and presenting these visualizations which also supports the interaction of the graphical mode with other modes is described.

The project undertakes research into multimodal interfaces to knowledge based systems (KBS).
Such as interface will allow the user to communicate with the underlying application KBS using natural language, direct manipulation and gesture, as well as a command language. It will also allow the application to communicate with the user through natural language and structured graphics. Natural language modules will be developed for English, French and Spanish. The ergonomic aspects of the interface will receive special attention in order to maximize its readability and efficiency and favour its use even in real time situations.
The interface will incorporate a dialogue management and mode selection system which will use knowledge of the specificites of individual modes, of the context of previous interactions, and of the application domain to determine the content of system output and select the most appropriate mode in which to present particular information. A user modelling module will interact with dialogue management, so that the system will react appropriately to different classes of users and individual users.
The interface developed is designed to be portable across a range of potential applications of Prolog based KBS.

Work so far has concentrated on designing the architecture and operating principles of the interface system and on preparing prototypes of each mode. A prototype was developed that demonstrates an initial implementation of the system's modules. The final phase concentrated on an extension of the system's intelligent interaction capabilities and on the exploitation of the research results in an industrial setting.
The interface allows the user to communicate with the underlying application KBS using natural language, direct manipulation and gesture, as well as a command language. It also allows the application to communicate with the user through natural language and structured graphics. Natural language modules are developed for English, French and Spanish. The English system is based on previous results and prototypes built during an ESPRIT I project (LOKI). The ergonomic aspects of the interface receive special attention in order to maximise user friendliness and efficiency and allow for use even in real-time situations.

The interface incorporates a dialogue management and mode selection system which uses knowledge of the specificities of individual modes, knowledge of the context of previous interactions, and knowledge of the application domain to determine the content of system output and to select the most appropriate mode in which to present particular information. A user modelling module interacts with dialogue management, in order to allow the system to react appropriately to different classes of users and different types of individual users.

The interface developed is designed to be portable across a range of potential applications of Prolog-based KBS. The project focuses on realistic problems and will be demonstrated and evaluated using a specific application, an expert system in computer network design. This application is also developed inside the project. A number of demonstrator prototypes of the interface integrated with the application will be produced in the course of the project.

Thème(s)

Data not available

Appel à propositions

Data not available

Régime de financement

Data not available

Coordinateur

BELG.INST.FOR MANAGEMENT
Contribution de l’UE
Aucune donnée
Adresse
KWIKSTRAAT, 4
3078 EVERBERG
Belgique

Voir sur la carte

Coût total
Aucune donnée

Participants (6)