. The main results are the following:
Result 1: A TRL5 smart camera-based solution for cabin crew efficiency and flight safety
The created TRL5 system contains two types of models for the AI component’s “brain”: (1) a DNN model that is used for the extraction of descriptive feature vectors from the mentioned image regions (cropped and pre-processed), and (2) a set of manifold-learning-based discriminant space models, used for further processing those feature vectors for the TTL-condition classifications stage. The former is trained “out-of-the-box”, i.e. it is deployed already trained for every installation and it is not expected to be modified/readjusted, “on-site”. The latter can be modified more easily (i.e. “on-site”), as it can be quickly retrained with the computational resources available in the AI-processor. The system is patent pending.
Result 2: Synthetic 3D simulations for camera integration and photorealistic data generation
We have built a tool that allows visualizing the 3D models within the cabin as if they were being observed from the cameras to be installed, simulating the kind of images that would be captured due to their characteristics along with the selected position, orientation, illumination conditions, etc. The methodology to create this kind of tool was published (
https://zenodo.org/record/4548650#.Y77UcBXMJD8(öffnet in neuem Fenster)).
Result 3: Large-scale dataset for training image content descriptors in the context of airplane cabins
To collect the necessary data, a cabin mockup was built, and synthetic data was generated using 3D graphics. The mockup was illuminated from three possible light sources: natural light from the room's windows, artificial light on the ceiling, and a spotlight beside the cabin window to mimic directional sunlight. We published part of the dataset (
https://zenodo.org/record/7524808#.Y77M7xXMJD8(öffnet in neuem Fenster)) to support research on object detection and scene understanding, specifically related to identifying the proper positioning of cabin luggage during taxi, take-off, and landing (TTL) operations.
Result 4: Synthetic-to-real domain adaptation algorithm for augmented dataset generation
We developed a methodology to exploit the gathered synthetic and real data to train more effectively the AI component. The methodology was published (
https://zenodo.org/record/7282478#.Y77UNRXMJD8(öffnet in neuem Fenster)).
Result 5: Image content descriptor algorithm for optimal detection of cabin components, luggage, subjects, and their visual relations
We developed a methodology to analyze images and describe their content by examining the relationships between various factors, including the presence or absence of specific elements, such as people or objects, and the positioning of these elements within the image. This methodology is currently under review for publication.
Result 6: Optimization techniques for efficient inference of deep neural network models
We developed a methodology to optimally deploy the image content descriptor algorithm onboard, processing the image streams captured by the required number of cameras to cover all the cabin areas. This methodology is currently under review for publication.
the tool we refer to under Result 2 is a tool with 3D models within the cabin; while in Result 6 we refer to a methodology to optimally deploy the image content descriptor algorithm onboard.
The dissemination activities were conducted during COVID 19 pandemic period so we haven't been able to achieved our main KPI.
