Periodic Reporting for period 2 - EARS (Acoustics-based drone navigation and interaction)
Période du rapport: 2022-03-01 au 2023-08-31
Autonomous multi-rotor aerial vehicles (MAVs) are an emerging technology, which has a large number of current and potential applications in a wide range of industries. These airborne vehicles are becoming growingly autonomous thanks to modern artificial intelligence technologies, with their navigation and interaction capabilities based predominantly on visual sensing. While vision navigation has attracted considerable attention, it suffers from a poor performance in low light, limited field of view, and direct sunlight and is vulnerable to occlusions. Acoustic sensing can complement and even replace vision in many situations, and it also benefits from lower system cost and energy footprint, which is especially important for small form-factor aircraft. While novel acoustic technologies based on phased microphone arrays are making their way into the Internet of Things and home automation markets, their use in MAVs is currently impeded by the strong self-noise generated by the drone propulsion system. Consequently, existing commercial and research aerial platforms have advanced vision capabilities, yet no acoustics. This project targets to change this situation, endowing drones with "ears". The proposed research aims at the development of novel machine learning-based algorithms and real-time systems for acoustic-based autonomous mapping, localization, and interaction of MAVs. One of the key ideas of the proposal consists of actively controlling and shaping the aircraft self-noise for the benefit of the navigation and interaction tasks, instead of considering it a harmful nuisance. Our end goal is to demonstrate a flying proof-of-concept system which, to the best of our knowledge, will be the first of its kind. While the primary goal of the project is very specific, achieving it will require a considerable amount of scientific and methodological innovation in modelling, signal processing, and machine learning that we expect to have a significant impact on broad domains.
The following tasks have been achieved as planned:
- Objective 1 (Theoretical analysis and physical modeling): We have built simulation models for single rotor free-space acoustic source, a model for multiple rotors with arbitrary phase modulation, and a differentiable geometric environment model. We currently have a functional differentiable forward model of a multi-rotor aircraft in an acoustic environment.
- Objective 2 (Algorithms and computational methods): We have shown a proof-of-concept of localization uncertainty reduction in a known environment by means of rotor phase modulation and started working on learning the optimal modulation. We also achieved a demonstration of close-loop adversarial attacks on SLAM systems and started exploring the interaction between acoustic SLAM and the aircraft control system.
- Objective 3 (System and infrastructure): We have built a platform for collecting data from a single non-flying rotor with exquisitely controlled excitation signals. First data sets were collected. We have also started building airborne prototypes of the acoustic acquisition and rotor modulation subsystems.
- Objective 1 (Theoretical analysis and physical modeling): We have built simulation models for single rotor free-space acoustic source, a model for multiple rotors with arbitrary phase modulation, and a differentiable geometric environment model. We currently have a functional differentiable forward model of a multi-rotor aircraft in an acoustic environment.
- Objective 2 (Algorithms and computational methods): We have shown a proof-of-concept of localization uncertainty reduction in a known environment by means of rotor phase modulation and started working on learning the optimal modulation. We also achieved a demonstration of close-loop adversarial attacks on SLAM systems and started exploring the interaction between acoustic SLAM and the aircraft control system.
- Objective 3 (System and infrastructure): We have built a platform for collecting data from a single non-flying rotor with exquisitely controlled excitation signals. First data sets were collected. We have also started building airborne prototypes of the acoustic acquisition and rotor modulation subsystems.
In the present report period, we showed for the first time in simulation that active modulation of the aircraft rotors leads to improved acoustic-based localization. We also developed a highly-accurate and fast free-space acoustic model for rotor noise based on real recorded data, and an acoustic simulation of environments. Both models are differentiable which, to the best of our knowledge, is entirely novel. Our next major modeling goal is to explore inverse problems with the said models, namely, devising optimal rotor modulation, optimal blade geometry, and optimal flight trajectories. On the system side, we are making notable progress in the construction of an airborne system with flexible and programmable control of rotor modulation and high-resolution acoustic acquisition capabilities. To the best of our knowledge, no such systems exist (at least, commercially).