SEnsorial awareNess System fOR obstacle detectIon And collisioN avoidanCE

Project Information

SENSORIANCE

Grant agreement ID: 785315

DOI

10.3030/785315

Project closed

EC signature date 14 February 2018

Start date 1 March 2018

End date 29 February 2020

Funded under

• SOCIETAL CHALLENGES - Smart, Green And Integrated Transport

Total cost

€ 1 572 750,00

EU contribution

€ 1 100 925,00

Coordinated by

MESUREX SL
Spain

Periodic Reporting for period 2 - SENSORIANCE (SEnsorial awareNess System fOR obstacle detectIon And collisioN avoidanCE)

Reporting period: 2019-03-01 to 2020-02-29

Summary of the context and overall objectives of the project

New rulings are being published enabling landing operations with Enhanced flight vision system (EFVS) under reduced visibility conditions, such as with no visible references like the approach, the threshold, or the runway edge lighting systems. Recently, for example, the US Federal Aviation Administration (FAA) enabled trained pilots in the use of EFVS to continue descending 100 feet below the Touchdown Zone Elevation (TDZE), and even perform a full landing procedure with no natural vision under specific conditions. This is leading to new advanced EFVS that meet the requirements to adapt to these new regulations.


The main goal of SENSORIANCE project is to develop a cost-effective system to provide combines information from various sources, including a compact camera –consisting of a cost effective, versatile, high-performance and highly-reliable optical system— and external sensors, distributed in the form of LRU modules. This information is then processed by a computer vision and image processing module to infer useful knowledge and event-reaction capabilities, aiding pilots during all phases of flight.

SENSORIANCE system will provide a complete enhance and synthesized images suite and it will allow to enhance the field of view, to reduce the risk of occurrence of an impact, to Enable air transport system optimization, to ensure compliance with safety standards and regulations, to use ROHs components, requiring low investment in devices (COTs).

Based on these general objectives, the technical objectives of SENSORIANCE are the following:
• To develop an affordable uncooled VIS/IR sensor.
• To develop software components that will consist of several modules to perform the following tasks: manage user input and console commands, present feedback messages to the user, control hardware modules, setup and maintenance automation and computer vision functionality.
• To develop a precise mechanical assembly of lenses that will meet the different focal lengths required by the system. Image enhancement capabilities with a good compromise on cost, performance and weight will be introduced.
• To design, develop and manufacture of mechanical casing and electrical circuits under RTCA/DO-160G for the TRL5 prototype to comply with airborne hardware requirements.
• To develop different modules for image acquisition, image compression and communications via the standard Ethernet or ARINC818 interfaces.
• To design, create test environments for visualization and simulation and perform the different electrical, mechanical and environmental tests specified in the RTCA/DO-160G.
• To design a system with a Mean Time Between Failure (MTBF) for the LRUs of 40.000 h.
• To provide a DRI (Data Requiremente Item) that contain a qualitative assessment of the System and/or LRU BIT capability.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

First of all, a study was carried out for the selection of optimal detectors spectral bands for compliance with the primary requirement of the project (the improvement of visibility in adverse weather conditions) and secondly, for the evaluation of existing technologies within each type of detector.

In order to assure the accomplishment of the requirements for the whole project, different detectors for each one of the bands of the electromagnetic spectrum were studied and evaluated for use in SENSORIANCE: silicon based detector for the visible and near infrared bands and uncooled LWIR detectors. To accomplish the mechanical requirements, several options according to the fabrication process and materials selection were studied.

Hardware and software modules of SENSORIANCE were defined. The pipeline inside SENSORIANCE follows a sequential processing order, where the environment is sensed via detectors and put together into a coherent video stream by their respective acquisition modules. These video streams are then passed on to the hardware processing modules which fuses all video streams into one combined, enhanced, and augmented multispectral video output. This final combined video output is then converted back by the hardware output modules to produce a standard signal that can be understood by the aircraft’s output devices (HUD or HDD).

Mesurex proceeded with the production of the physical prototype of the system. Imaging sensors (VIS, NIR and LWIR) were encapsulated into the mechanical and optical envelop. Imaging tests were performed to ensure that the device is capable of imaging according to the specification and needs of the system, producing the correct output, and transmitting information through the fibre optic data channel. These imaging tests have not required specific environments, only that the field of view is large enough, and the scene complex enough to test the capabilities of the resulting assembly in terms of focal length, focus, contrast, dynamic range, signal-noise ratio, frame rate, etc.

Environmental tests were performed to withstand the final operating condition without failing or breaking. Also, electromagnetic interference (EMI) and electromagnetic compatibility (EMC) test were performed on the system to prevent errors caused by the effect of close sources of electromagnetic radiations to prevent causing damages and interference to other devices in the aircraft.

A laboratory unit acceptance review (LUAR) has been issued, accepting that the equipment standard that is used in the laboratory is compliant with all specifications.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Nowadays, there are a number of commercial EFVS systems in service but many still utilize mid-wave IR sensors (operates in the 1-5 μm band) that are cryogenically cooled by liquid nitrogen refrigeration to very low temperatures to gain the sensitivity to see the minute-temperature differences required to image the world via thermal sensing. The cryo-cooled units are heavy, expensive, require pesky maintenance and involve significant cooldown times before they can operate.

SENSORIANCE aims at providing a solution to these issues while, at the same time, using more affordable technology than the ones already available in the market by combining information from multispectral IR and visual range sensors. Tha advantages of SENSORIANCE are:

- Increased safety levels in low visibility conditions and bad weather conditions.
- Lower operations costs for aircrafts operators. A reduction in weather related delays provides an operator with the potential for substantial savings.
- Reduced fuel burn and environmental footprint (CO2, NOx, noise). The fuel savings is achieved by fewer flight delays, fewer missed approaches, fewer flights forced to an alternative destination, and a decrease or elimination in hold times.

The main benefits of SENSORIANCE versus other products are the following:

-LWIR resolution of 1024 x 768 pixels vs 640 x 480 pixels in current products.
-VIS/NIR resolution of 1920 x 1080 pixels vs 1280 x 960 pixels in current products.
-3,950 Kg of weigth vs 5,5 Kg of similar products (Elbit Systems EFV).
-Fiber optic transmission vs wired transmission in current products.
-75% cost saving (on average) vs current products.
-50% size decrease (average) vs current products.
-50% size decrease (average) vs current products.
-SENSORIANCE has software/algorithms to identify and adapt to the illumination condition of the environment, to detect objects and events and to focus on Region of Interest (ROI) not available in current products.
-0,06 Kg of fuel reduction per hour vs current products.

SENSORIANCE prototype modular system