Periodic Reporting for period 1 - I-SPOT (Intelligent Ultra Low-Power Signal Processing for Automotive)
Période du rapport: 2020-11-01 au 2022-10-31
While traditional cars relied solely on the senses of their drivers – mostly their eyes and ears –, the car of today is equipped with more and more sensors. Self-driving cars come within reach by equipping cars with their own set of eyes, in the form of LIDARs and cameras. Yet, to become truly aware of their environment, such smart cars are still missing an invaluable input on which we, as humans, rely strongly: the acoustic information. I-SPOT targets the addition of acoustic sensing technology to cars, to bring enhanced environmental awareness.
This acoustic information complements the information from other sensory technologies. In active (drive) mode, this will give information about nearby emergency vehicles, accidents, passing cars, etc., which are currently causing major disruptions of autonomous and computer assisted driving. Moreover, information on weather conditions, or mechanical car wear or failure is present in the acoustic signal. More than just detection of the nature of the sound, also the direction can be derived, and this even for visually occluded sources. In passive (park) mode, information can be obtained on car damaging, theft, or nearby critical events (e.g. cry for help). The acoustic sensor can as such form the low-cost wake-up trigger to direct the more power-hungry camera system to activate and point in a specific direction.
The goal of I-SPOT is to drive this domain from two different angles:
As its technical contribution, I-SPOT aims to enable to sense, localize and analyse environmental audio signals during the active and passive car mode (namely drive and park mode) by innovating at:
• the efficient placement of audio sensors on the car body to improve the received signal quality,
• the development of low footprint signal processing technologies for automotive acoustic signal characterization and localization
• the design of a smart, adaptive, ultra-low power hardware that can be always-active, also when the car is switched off.
This acoustic information complements the information from other sensory technologies. In active (drive) mode, this will give information about nearby emergency vehicles, accidents, passing cars, etc., which are currently causing major disruptions of autonomous and computer assisted driving. Moreover, information on weather conditions, or mechanical car wear or failure is present in the acoustic signal. More than just detection of the nature of the sound, also the direction can be derived, and this even for visually occluded sources. In passive (park) mode, information can be obtained on car damaging, theft, or nearby critical events (e.g. cry for help). The acoustic sensor can as such form the low-cost wake-up trigger to direct the more power-hungry camera system to activate and point in a specific direction.
The goal of I-SPOT is to drive this domain from two different angles:
As its technical contribution, I-SPOT aims to enable to sense, localize and analyse environmental audio signals during the active and passive car mode (namely drive and park mode) by innovating at:
• the efficient placement of audio sensors on the car body to improve the received signal quality,
• the development of low footprint signal processing technologies for automotive acoustic signal characterization and localization
• the design of a smart, adaptive, ultra-low power hardware that can be always-active, also when the car is switched off.
* Both ESRs have been hired and started.
* ESR1 has:
- Assessed available sound source datasets
- Started the implementation of a new dataset generation framework, targeting outdoor car scenarios.
* ESR2 has:
- performed an extensive analysis op sound source locazation algorithms
- Implemented a signal processing training and performance assessment framework to train sound source location models.
- Optimized an SRP-PHAT + NN based model for low computational footprint for embedded processing
- A first publication is currently in the making
* ESR1 has:
- Assessed available sound source datasets
- Started the implementation of a new dataset generation framework, targeting outdoor car scenarios.
* ESR2 has:
- performed an extensive analysis op sound source locazation algorithms
- Implemented a signal processing training and performance assessment framework to train sound source location models.
- Optimized an SRP-PHAT + NN based model for low computational footprint for embedded processing
- A first publication is currently in the making
* ESR1 has created the first acoustic dataset generator for outdoor sound source localization and identification cases
* ESR2 has created the most low footprint SRP-PHAT based sound source localization algorithm, and analyzed it extensively from both a performance and hardware cost point of view.
Towards the end of the project, we will:
* Merge these 2 initial contributions from ESR1 and ESR2
* Use that towards actually implementing an on-board (in the car) sound source localization and identification system
Towards society, this will benefit the safety of cars:
In drive mode, this will give information about nearby emergency vehicles, accidents, passing cars, etc., which are currently causing major disruptions of autonomous and computer assisted driving. Moreover, information on weather conditions, or mechanical car wear or failure is present in the acoustic signal. In passive (park) mode, information can be obtained on car damaging, theft, or nearby critical events (e.g. cry for help)
* ESR2 has created the most low footprint SRP-PHAT based sound source localization algorithm, and analyzed it extensively from both a performance and hardware cost point of view.
Towards the end of the project, we will:
* Merge these 2 initial contributions from ESR1 and ESR2
* Use that towards actually implementing an on-board (in the car) sound source localization and identification system
Towards society, this will benefit the safety of cars:
In drive mode, this will give information about nearby emergency vehicles, accidents, passing cars, etc., which are currently causing major disruptions of autonomous and computer assisted driving. Moreover, information on weather conditions, or mechanical car wear or failure is present in the acoustic signal. In passive (park) mode, information can be obtained on car damaging, theft, or nearby critical events (e.g. cry for help)