Skip to main content

tHErmaL vIsion AUgmented awarenesS

Periodic Reporting for period 2 - HELIAUS (tHErmaL vIsion AUgmented awarenesS)

Reporting period: 2019-04-01 to 2020-03-31

The new ways of driving and use a vehicle as expected in the scope of Smart Mobility asks for reliableand affordable versatile perception systems. It is now clear that the interaction between the vehiclesand its environment (in and out of cabin) has to be improved. As the automotive market has a strongstrategy towards the development of autonomous vehicles, this need even more important in the caseof autonomous driving for which the four mandatory steps (starting with “hands on” and ending with“hands off”, “eyes off” and “minds off”) are full of technical hurdles, barriers, obstacles.
Perception systems need to be accurate and reliable both for what happen in the cabin and out ofthe cabin. The driver and passenger monitoring needs to be improved as well as the environmentsurrounding monitoring, in all light and weather conditions. For both of these topics, the extension ofcurrent perception systems (mainly based on visible imaging, LIDAR and RADAR detection) to thethermal sensing is a must. Thermal sensing, especially in the Long Wavelength Infrared or LWIR (=6 to 14 μm) bandwidth, provides valuable additional information and has to be considered as a mustfor the next generation of L4 (‘’Eyes off’’) and L5 (‘’Mind off’’) autonomous driving.
In this context, the HELIAUS project aims to deliver breakthrough perception systems for in-cabinpassengers monitoring, as well as for the car surrounding by developing smart thermal perceptionsystems that extend the current systems to the LWIR bandwidth.

The HELIAUS project focuses on the Sense and Think part of the perception process for both in andout-of-cabin applications.
The HELIAUS project integrates high performance and low cost LWIR thermal systems, images / dataprocessing units and software (SW) applications into smart thermal perception systems compatible tosuch applications. The thermal perception systems can be used solely or in combination with otherssensors commonly used for the applications, as vision sensors for in-cabin monitoring or LIDAR forout-of cabin application, leading to an augmented awareness and reliability.
From general point of view, the main objectives of the HELIAUS project and work to be carried outare as follows:
- to specify, develop, test and validate the thermal perception systems, for in-cabin and out-of cabinapplication,
- to develop cutting-edge and cost effective technologies leading to low cost, high performance LWIRmodule,
- to quantify the valuable addition of the thermal sensing into the current systems and future systems,
- to promote the benefit of such systems in the future autonomous vehicles, in particular, and for smartmobility in general.
Despite COVID 19 situation and planning disruption the HELIAUS project follow their objectivesand major technological blocks are on track.
The Heliaus project is divided in 3 main areas:
- Project and innovation management
- Disruptive technologies development
- Demonstrator building & uses-cases evaluation

Project and innovation management actions achieved:
- Set up an efficient project organization with project planning, collaborative platform, monthly calls & reporting to overpass COVID pandemic delays
- Support innovation by creating communication tools as graphic charter and website
- Support dissemination activities targeted by the project leading to value creation

Disruptive technologies development actions achieved:
- Innovative thermal sensor and packaging design to improve cost and robustness
- Innovative process leading to reduce cost and to supply first prototypes
- Laboratory testing of first prototypes
- Design and prototypes innovative thermal image processing algorithm
- Specifications of a SW/HW Platform for thermal image processing
- Testing different ways of software embedding
- Design and prototype lenses for innovative sensors
- Design and test new lens manufacturing and assembly processes for large volumes productionleading to lower cost
- Demonstrator building & uses-cases evaluation actions achieved
- Release specifications of the demonstrators
- Demonstrators supply to HELIAUS partners
- Thermal IR (TIR) image simulator developed for generation of physically-realistic TIR images witha large number of parameters
- Development of a multi-sensor data acquisition system for in and out of cabin
- Dataset record and labeling for Computer Vision
- Training and performance metric of Computer Vision with thermal images
The major concrete expected impact items of the HELIAUS project are threefold and as follows:
- technological impact items expected from technology developments:
- Sensor: cost reduction thanks to innovative design and process
- Thermal image processing: optimize for the applications, markets and costs targeted
- Optics: cost reduction thanks to innovative design and process
- innovative IR modules and Smart IR modules to find the best compromise of performance and cost
- applicative impact items expected from in-cabin and car environment perception developments andevaluations
- Specification of the system
- Demonstration of computer vision thermal-based ADAS applications improvement