Skip to main content

tHErmaL vIsion AUgmented awarenesS

Periodic Reporting for period 1 - HELIAUS (tHErmaL vIsion AUgmented awarenesS)

Reporting period: 2019-04-01 to 2020-03-31

Infrared technologies are part of photonic field which includes all technologies that use light, create light, detect light, or modify light. Photonics is one of the six key enabling technologies recognized by the European Commission, and is well placed to address our most pressing societal challenges. Photonics will have an influence similar to the semiconductor industry. In particular, infrared technologies are relevant to the automotive industry, and to invite the automotive manufacturers and component suppliers to engage with the photonics value chain. With the emergence of new, often disruptive, forms of personal mobility, the automotive industry is facing a drastic change in its role and relationship with consumers, end-users and with society in general. Whether they are using their own vehicle or sharing one, drivers will demand a better driving experience for themselves, their passengers and other road users: quality and safety, simplified driving (i.e. increasingly autonomous and connected cars), air quality, and the contribution to combating global warming.
In this context, the HELIAUS project aims to deliver breakthrough perception systems for in-cabin passengers monitoring, as well as for the car surrounding by developing smart thermal perception systems that extend the current systems to the LWIR bandwidth.

The HELIAUS project focuses on the Sense and Think part of the perception process for both in and out-of-cabin applications.
The HELIAUS project integrates high performance and low cost LWIR thermal systems, images / data processing units and software (SW) applications into smart thermal perception systems compatible to such applications. The thermal perception systems can be used solely or in combination with others sensors commonly used for the applications, as vision sensors for in-cabin monitoring or LIDAR for out-of cabin application, leading to an augmented awareness and reliability.
From general point of view, the main objectives of the HELIAUS project and work to be carried out are as follows:
- to develop cutting-edge and cost effective technologies leading to low cost, high performance LWIR module,
- to specify, develop, test and validate the thermal perception systems, as depicted in
Figure 2 first prototypes for in-cabin and out-of cabin application,
- to quantify the valuable addition of the thermal sensing into the current systems and future systems,
- to promote the benefit of such systems in the future autonomous vehicles, in particular, and for smart mobility in general.
Work performed over the RP1 period for WP1 is:
- Set up an efficient project organization
- Define the project calendar
- Deploy a collaborative platform
- Deployment of monthly calls & reporting
- Finalize the Consortium Agreement

Work performed over the RP1 period for WP2 is:
- Provide a communication kit
- Defin a graphie charter
- Setup the HELIAUS website (
- Organize the website release social media campaign
- Update the communication & dissemination plan

Work performed over the RP1 period for WP3 is:
- Design the first pixel readout architecture
- Design and process the preliminary 8.5um pixel
- Optimize bolometer process to the B2OX technology
- Optimize WLP process to the B2OX technology
- Provide the first B2OX wafers

Work performed over the RP1 period for WP4 is:
- Benchmark ISP chips and algorithms for IR correction
- Release super resolution algorithm
- Specifications of a SW/HW Platform for IR Image Processing

Work performed over the RP1 period for WP5 is:
- Release of first design for WP8 applications
- Provide athermalization study
- First trials on plasma sintering manufacturing

Work performed over the RP1 period for WP6 is:
- Release specitifcations of the demonstrators
- Provide demonstrators to HELIAUS partners
- DEM1 packagin developments

Work performed over the RP1 period for WP7 is:
- Build IR pipeline for basic thermal imaging (driver monitoring)
- Build of a data acquisition system
- Thermal IR (TIR) image simulator developed for generation of physically-realistic TIR images with a large number of parameters
- review of state-of-art processing/analysis of thermal imaging data
- Several publication in the submission process

Work performed over the RP1 period for WP8 is:
- Definition of the dataset specifications
- Integration of DEM-0 on the car.
- Development of a HW concept for LWIR and VISIBLE camera hardware
- Developments of algorithms for “On Road Object Detection”
The major concrete expected impact items of the HELIAUS project are threefold and as follows:
- technological impact items expected from technologies WPs developments (from WP3 to WP5),
- innovative IR modules and Smart IR modules expected from WP6,
- applicative impact items expected from in-cabin (WP7) and car environment perception (WP8) developments and evaluations.