Skip to main content

Pilot Eye Gaze and Gesture tracking for Avionics Systems using Unobtrusive Solutions

Objective

As the capabilities of cockpit systems increase, the way in which the pilots interact with them evolves. At the same time, flight management concepts and tools continue to increase in complexity, as well as flight and non-flight data streams, which consequently challenges the pilots’ abilities to cope with the information they need to be aware of, and exerts pressure on their cognitive states.

In in framework of the development of new Human System Interfaces carried out in WP1.2.4 the Clean Sky 2 SYSTEMS ITD, the PEGGASUS project proposes an approach to bring innovative smart vision systems to cockpits. The system we propose to develop is a crew interface and monitoring system implementing eye gaze, face, and gesture tracking technologies to bring new functionalities to Human-Machine Interfaces in the cockpit. It consists in a set of NIR-sensitive cameras powered by Computer Vision and Machine Learning algorithms running on a central processing unit. Three prototypes will be developed from a standalone lab prototype, to a TRL5 model integrated in the topic leader’s “cockpit demonstrator”. The system design shall be compliant with aircraft cockpit installation, the operational environment and pre-certification requirements. The requirements defined at the beginning of the project with the topic leader’s inputs, will be verified after its successful integration into the “cockpit demonstrator” avionics.

PEGGASUS brings together partners able to cover the complete value chain of innovative product development: SERMA Engineering (FR), an OEM for aeronautical equipment, CSEM (CH) an RTD with proven experience in innovative technologies for aeronautics, ETHZ (CH) represented by the Geogaze lab that conducts cutting edge research with eye tracking technology, particularly for the use in aviation training, and Swiss (CH), an airline and potential end user who will provide expertise in crew interfaces and monitoring.

Coordinator

CSEM CENTRE SUISSE D'ELECTRONIQUE ET DE MICROTECHNIQUE SA - RECHERCHE ET DEVELOPPEMENT
Net EU contribution
€ 562 185,58
Address
Rue Jaquet Droz 1
2000 Neuchatel
Switzerland

See on map

Region
Schweiz/Suisse/Svizzera Espace Mittelland Neuchâtel
Activity type
Research Organisations
Other funding
€ 562 185,58

Participants (4)

SERMA INGENIERIE
France
Net EU contribution
€ 130 736,59
Address
Rue De L Aussonelle Au Village
31700 Cornebarrieu

See on map

Region
Occitanie Midi-Pyrénées Haute-Garonne
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 130 736,59
EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH
Switzerland
Net EU contribution
€ 218 683,98
Address
Raemistrasse 101
8092 Zuerich

See on map

Region
Schweiz/Suisse/Svizzera Zürich Zürich
Activity type
Higher or Secondary Education Establishments
Other funding
€ 218 683,98
SWISS INTERNATIONAL AIR LINES AG
Switzerland
Net EU contribution
€ 36 592,60
Address
Malzgasse 15
4052 Basel

See on map

Region
Schweiz/Suisse/Svizzera Nordwestschweiz Basel-Stadt
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 36 592,60
Third-party

Legal entity other than a subcontractor which is affiliated or legally linked to a participant. The entity carries out work under the conditions laid down in the Grant Agreement, supplies goods or provides services for the action, but did not sign the Grant Agreement. A third party abides by the rules applicable to its related participant under the Grant Agreement with regard to eligibility of costs and control of expenditure.

DEUTSCHE LUFTHANSA AKTIENGESELLSCHAFT
Germany
Net EU contribution
€ 2 625,00
Address
Venloer Strasse 151-153
50672 Koln

See on map

Region
Nordrhein-Westfalen Köln Köln, Kreisfreie Stadt
Activity type
Private for-profit entities (excluding Higher or Secondary Education Establishments)
Other funding
€ 2 625,00