Human fatigue is a serious issue affecting the safety of the traveling public in all modes of transportation. There is evidence that fatigue reduces the capacity for situational awareness, impairs decision making and slows down reaction time. In the safety-pioneering aviation sector, crew monitoring is seen as key to maintaining safety in case of incapacitation, stress, or exhaustion of crew members and for reduced crew operations.
The HIPNOSIS consortium, led by CSEM, has provided the hardware and software implementation of smart sensors to monitor cognitive states of pilots, such as drowsiness, which will allow a great leap in flight safety in the next generation cockpit. The HIPNOSIS approach is based on two sensing systems:
- A vision-based system integrated in the cockpit dashboard to continuously monitor the pilots in the cockpit and detect behavior linked to drowsiness in real time.
- A smart wristband to sense several bio signals through advanced optical sensing technologies, before, during and after the flight, in order to detect physiological measures linked to drowsiness.
The goal of such a user-centric approach is to improve fatigue-related safety and situational awareness and support disruptive cockpit operations. In order to fulfill the abovementioned objectives, HIPNOSIS brings together the latest artificial intelligence (AI) and computer vision techniques as well as optical sensing technologies with aeronautics expertise, contributing to the advent of next-generation cockpits. For such a monitoring system to be successfully adopted, an extremely high level of accuracy and robustness is required, considering aviation-specific environmental factors such as extreme lighting conditions and vibrations.
To design such a tailor-made solution for the aviation industry, the consortium partners worked in close collaboration with the topic leader Honeywell Aerospace, a leader in aerospace development and the manufacturer of avionics solutions. CSEM developed the vision system and machine learning algorithms for detecting behavioral features linked to drowsiness, addressing the unique challenges and requirements in the aeronautic context. In addition, CSEM has developed a smart wristband, based on optical sensing technologies, which allows direct on-body analysis of a pilot’s bio signals. SERMA Ingénierie was responsible for developing the HIPNOSIS vision system hardware, particularly the near-infrared illuminators (in compliance with eye safety regulations) and a system for synchronizing them with image capture, as well as integrating the system into the cockpit simulator. French startup Innov+, which already commercializes similar solutions for the automotive industry, contributed to the development of the HIPNOSIS software API.
