Solutions for voice interaction towards natural crew assistant

Informations projet

VOICI

N° de convention de subvention: 785401

DOI

10.3030/785401

Projet clôturé

Date de signature de la CE 20 Février 2018

Date de début 1 Mars 2018

Date de fin 29 Février 2020

Financé au titre de

• SOCIETAL CHALLENGES - Smart, Green And Integrated Transport

Coût total

€ 1 250 750,00

Contribution de l’UE

€ 1 111 250,00

Coordonné par

SINTEF AS
Norway

Periodic Reporting for period 2 - VOICI (Solutions for voice interaction towards natural crew assistant)

Période du rapport: 2019-03-01 au 2020-02-29

Résumé du contexte et des objectifs généraux du projet

"VOICI has aimed to develop a natural crew assistant – designed to process natural spoken requests from the crew members and help them perform complex actions. Such an assistant can reduce the cognitive workload associated with the increasingly complex human-machine interfaces in modern aircraft cockpits.
VOICI's main goal has been to demonstrate the natural crew assistant at Technology Readiness Level 3, i.e. experimental proof of concept. This includes speech capture, speech recognition and interpretation, and speech synthesis for dialog with the crew. An Audio Evaluation Environment has been established, for developing and testing the crew assistant under realistic audio/noise conditions.
The project is part of the ""extended cockpit"" work package under Clean Sky 2's ""Systems"" technology demonstration. By reducing crew workload VOICI contributes to optimization of operations; flight safety, crew awareness, better maintenance, reduced cost of operations and generally higher efficiency and lower stress.
The consortium comprises 4 partners: SINTEF (Norway, R&D institute, coordinator), sensiBel (Norway, SME), Multitel (Belgium, R&D institute), Acapela (Belgium, SME).
Thales (France, Clean Sky 2 partner) is Topic Manager.
The project has run 24 months, ending February 2020. By the project end the main goal has been achieved - providing proof of concept demonstration of the crew assistant for two use cases selected by the Topic Manager. The project has contributed to innovation and job creation by improving component technologies, demonstrating the concept and establishing new cooperation within the consortium and with the Topic Manager."

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

VOICI has established an Audio Evaluation Environment, emulating a case selected by the Topic Manager: a Dassault Falcon 2000 cockpit. This includes a geometrical cockpit model and calibrated noise recording, split into multiple channels so as to represent multiple noise sources. Hence, a realistic noise and reverberation environment has been created, including its directional properties. A digital version of the environment has been established by measuring impulse responses between multiple loudspeakers and multiple microphones.

Two methods of speech capture have been investigated: A pilot's headset using a close-talk microphone and an ambient microphone array. Captured speech has been processed by stand-alone speech/dialog processing functions, i.e. suitable for embedding in a cockpit without connection to external, e.g. cloud-based services. The processing includes speech enhancement, voice activity detection, context dependent speech recognition, dialog system and text-to-speech for the noisy environment.

Due to the high noise levels, the close-talk microphone has been found to be the preferred technology for speech capture. Still, arrays of SME partner sensiBel's very low noise optical microphones were shown to produce significant noise suppression when used with modern beamforming algorithms. VOICI has contributed to sensiBel's microphone technology development, representing a significant project exploitation potential both in aviation/transport and in a much wider market.

The speech processing engine has demonstrated 5% Word Error Rate for cruising flight conditions, relying on an context -dependent language model and deep learning approaches for training. The dialog system was built to additionally include touch-screen input, together with external aviation information such as airport and weather data. It has been designed for easy inclusion of extended skills in terms of new stories, intent, entities and actions. Speech based feedback to the pilot was implemented by SME partner Acapela's text-to-speech function optimized for the noisy environment.
The total crew assistant has been called SOLO. The Topic Manager has signalled that an Implementation agreement will be negotiated. Acapela has elaborated plans for exploitation of both the total concept and the optimized text-to-speech function.

Dissemination items of the project have been identified. Due to the relatively short duration of the project, most of the dissemination-plan will be implemented after the end of the project. This will take place in close dialog with the Topic Manager.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

"VOICI's total concept has contributed to the state-of-the-art in cockpit-embedded speech processing and dialog systems – tailored to the aviation scenario and high noise levels. Similarly, the project has contributed to the state-of-the-art in component technologies of the SME partners; very low noise optical microphones and noise-optimized text-to-speech technology.

Through a reduced crew workload VOICI will contribute to Clean Sky 2's ""extended cockpit"" goals; optimization of operations, flight safety, crew awareness, better maintenance, reduced cost of operations and generally higher efficiency and lower stress."

SOLO architecture

Embedded, standalone natural crew assistant

Microphone array with front end electronics. Endfire configuration example.

Audio evaluation environment built from sheet plywood and transparent acrylic