The following work has been performed:
WP1- “Cockpit Operations & Case Studies”: , the Operational Concept for the full project has been finished with a definition of the use case and the E-Pilots simulation framework functionalities. A deep state of the art review on flight deck Cognitive Computing supporting tools has been the baseline to identify present gaps and opportunities to improve PF performance while preserving safety in a SPO framework. A Roadmap explaining the need on further research for a safety deployment of cognitive computing supporting services in the flight deck has been documented.
WP2- “Cognitive Sensor Net”: different physiological sensors has been considered and tested to characterize the pilot cognitive state together with the algorithms to translate their output into a cognitive scale. An architecture integration of the sensor with the Rolls Royce Future Flight Simulator provided by Cranfield University for validation purposes has been described together with a set of experiments for validation purposes. The original architecture has been upgraded to perform the experiments in an A-320 cockpit simulator. In this WP, it has been implemented 3 different serious games in which the human operator creates Prospective Memory Items for data gathering purposes.
WP3, “Cognitive Computing”: The methods and development of the predictive algorithm to improve the pilot situational awareness by informing about the probability of a hard landing have been described, implemented and tested using data from Flight Monitoring System (FMS) of 377,446 flights. The implementation of a ML algorithm to monitor the cognitive status of the PF has been implemented using EEG and ECG data. The results achieved in WP3, confirms that ECG is more stable and results can be more easily transferred at the cost of a limited capability for discriminating among different cognitive states and detecting states not related to stress and heart rate (like mental collapse). Also, it has strong dependency on the baseline state of the subject which suggests that the best use of ECG could be the detection of relative mental alterations across the flight. On the contrary, EEG is stable to discriminate different mental states at the cost of a lower task transfer power and a demand of a larger data set for training models. Data has been collected from serious games exercises and A-320 Cockpit simulations.
WP4 - “Socio-Technological Simulation Model”: It has been analysed the Standard Operating Procedures (SOP) from the “Approach Briefing” to “Reaching Minima” to specify the different PF and PM tasks using the FRAM formalism. This model will be the baseline to identify the need of cognitive computing supporting tools avoiding prospective memory items which impact on the pilot performance. The causal model has been implemented in FRAM formalism and adapted to SPO by assuming PM uncapacitated. The models has been validated by means of different experiments in an A-320 cockpit simulator. The model provides excellent tool to predict the PF workload, and allows the design of mitigation mechanism to avoid the creation of prospective memory items when PF is attending concurrent actions.
WP5 - “Verification and Validation” : It has been analysed the functional and non-functional requirements of the simulation framework to support the use case exercises, in order to get a better understanding of the barriers and enablers of cognitive computing supporting tools in the knowledge-based pilot flying tasks. The validation exercises for the ML2 and the socio-technical model were implemented in an A-320 cockpit simulator. An experienced line pilot was monitored with the EEG and ECG devices when flying the different scenarios.
WP6 - “Dissemination, exploitation and IPR management”, a detailed dissemination plan has been defined, with the high-level communication objectives, the target stakeholders, messages to be delivered, channels to be used, and activities to be performed.Worthwhile to emntion the publication of 5 scientific papers in Q1/Q2 journals.
Overview of exploitable results:
1.- Socio-Technological Simulation Framework.
2.- Machine Learning to predict a hard landing.
3.- Machine learning to monitor Huamn operator Cognitive status.
