Periodic Reporting for period 1 - Co2Team (Cognitive Collaboration for Teaming)
Reporting period: 2019-01-01 to 2020-06-30
Co2Team aims to create an artificial intelligence teammate to collaborate with the pilot built on top of the state-of-the art of human factors and a pilot-centred approach. This AI based on Cognitive Computing Technologies (CCT) allows a natural communication with the pilot. In other words, the pilot will be able to communicate with the intelligent teammate as he/she will do with a human teammate, by voice, gestures, gaze. An innovative bidirectional and multimodal communication solution will be tested. The pilot remains the captain on board and pilot-in-command, the CCT provides information to the pilot for better data driven decision making. This intelligent teammate will be able to consider the pilot and the context to best adapt information, communication and collaboration.
Co2team approach is based on 5 objectives:
• Identify the technical and methodological potential of cognitive computing in the different areas of application to cockpit operations
• Push up the role of the pilot in a collaborative relationship with very complex systems, with a Human-centred approach, the « Man in the Loop » concept for an optimized allocation of tasks and roles
• Construct an inventory of cognitive technologies and human to identify the potential of cognitive computing in cockpit operations
• Demonstrate the added value in flight safety and pilots’ acceptability of a cognitive collaborative teaming
• Develop a roadmap for an optimized and accepted collaboration pilot/system supported by cognitive computing technologies
Co2Team consortium is composed of five main partners. Each one of them contributes to the project with unique expertise: experts in human factors in aeronautics and cognitive technologies (Bordeaux INP-CATIE), experts in Artificial Intelligence (DFKI) and expert pilots (ECAIR,AKIANI). The project started on January 2019 and it will finish on December 2021. More information and news available at https://www.catie.fr/en/co2team-en/.
As a first step, the Co2Team consortium produced a state of the art (SoA) of cognitive computing and its use in different application domains and described the difference with a normal AI. Both use the same machine learning or deep learning technologies and can solve very complex problems in a short period of time, but their approach and objective are very different. In addition to focusing on natural communication, CCT supports human decision making while AI aims to solve problems without human intervention. Thus, the pilot's skills are increased rather than adding more AI-based automation. The advantage of CCT is that it can process a large amount of data, monitor and cross-reference it, to provide the pilot with reliable and understandable information. To learn, these technologies need a large amount of data, but the intelligent teammate must be able to help the pilot in abnormal situations and therefore learn with little data. A deliverable is therefore dedicated to the learning of rare events and intelligibility of Machine Learning.
The integration of an intelligent teammate in a cockpit requires a thorough analysis of the human factors involved. The Co2team consortium worked for eight months on Human Factors (HF) involved in a Human intelligent Machine Team, in parallel with the SoA on cognitive computing. We looked at past and present research and attempts in different fields of application. The knowledge acquired, with SoA in parallel, on CCTs has made it possible to extend research and to move away from the state of the art on human factors related to system automation. We provided a state of the art of the human factors involved in a hybrid team (Human-Cognitive Computing Teammate) and how to form this team that communicates, shares knowledge, information, collaborates and trusts each other to ensure flight safety at the highest level. As we had the objective of flight safety, we reviewed human factors recommendations and obligations and how they could be updated and implemented for SPO with a CCTeammate. In addition we interviewed pilots to make an in-depth analysis of all the tasks and knowledge required by pilots to carry out their duties and get their views on the implementation and interactions with the CCT and their ideas and recommendations for better acceptability.
After submitting those four deliverables, Co2Team consortium completed the second milestone of the project and launched a review of Cognitive technologies for bidirectional Human-System collaboration and the definition of the use case.
Co2Team consortium identified a specific use case and a very complex and risky flight scenario designed with pilots and human factors experts to challenge the HiMT. The scenario includes several flight phases (descent, approach, Go Around, and landing) the most accident prone according to Airbus and Boeing figures. We submitted and discussed the use case scenario with Airbus, Dassault and Thales who found it full of interest, useful and in relation to their expectations.
Clean Sky 2 Cognitive Computing topic is specifically designed to contribute to the following expected impacts:
• Intuitive and multimodal pilot interfaces
• Increased and more efficient communication with the ground
• Aircraft status, pilot behaviour monitoring
• Systems management improvement
• Reduced pilot workload
• Augmented vision systems with head-up/head Down displays
Co2Team’s objective of contributing to these expected impacts is specifically addressed in the four technical work packages, completed and in progress: WP1, State of the art about cognitive technologies, WP2 State of the art about man-machine teaming, WP3 Cognitive technologies for bidirectional Human-System collaboration, WP4 Case studies and demonstration in a flight simulator.