Intelligent Scene Sensing and Analysis in Underwater Environments

Informations projet

iSEAu

N° de convention de subvention: 101030367

DOI

10.3030/101030367

Projet clôturé

Date de signature de la CE 4 Mai 2021

Date de début 1 Mars 2022

Date de fin 30 Decembre 2024

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 165 085,44

Contribution de l’UE

€ 165 085,44

Coordonné par

ETHNICON METSOVION POLYTECHNION
Greece

Periodic Reporting for period 1 - iSEAu (Intelligent Scene Sensing and Analysis in Underwater Environments)

Période du rapport: 2022-03-01 au 2024-02-29

Résumé du contexte et des objectifs généraux du projet

Reliable and detailed underwater scene sensing, and analysis has a fundamental role in numerous underwater activities with scientific and economic interest. In fact, blue economy in 2020 amounted to €218 BN (GVA) and counts more than 5 million jobs in EU. At the same time, intensive research work focuses in marine and underwater environments including the study of climate change and its impacts, marine biology, underwater archaeology, marine geology, and marine renewable energy.

The main objective of iSEAu is to advance the scene sensing and perception capabilities in underwater environemnts by employing state-of-the-art machine learning methods, and using combinations of conventional (RGB) and non-conventional cameras, like multispectral and single photons cameras (SPCs).

iSEAu aims to bring together the fields of computer vision, machine learning and remote sensing for optimally addressing the underwater visual sensing challenges. The project objectives address this challenge in two
levels. The first concerns the development of methods for reducing the geometric and radiometric distortions introduced by the water through learning-based methods, and the development of methods based on transient imaging for perception in challenging visibility conditions. The second level concerns the adaptation and enhancement to the underwater domain of state-of-the-art methods for image-based extraction of structural and semantic information, and their field-testing considering representative application scenarios.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

The iSEAu project introduced significant advancements in underwater imaging and analysis through the development of advanced AI-driven methodologies. It successfully integrated deep learning, physics-based modelling, and specialized imaging techniques to enhance underwater image quality and enable intelligent scene understanding. Key achievements include the creation of StreamUR for real-time image restoration, MD2IP for training-free multispectral demosaicing, and a deep learning-powered system for autonomous gas bubble detection. Field deployments, notably in the deep hydrothermal field of Kolumbo underwater volcano near Santorini, Greece, and at the coastal hydrothermal field of Palaiochori in Milos, Greece, validated these methods, demonstrating their robustness in real-world environments. Furthermore, the project advanced low-power AI models for embedded systems, enabling their use in platforms with limited computational budget, as ROVs, AUVs and benthic stations. The work of the project led also to contributions in affine domains like satellite imaging and radiation mapping.

Overall, the results of the iSEAu project confirm the feasibility and effectiveness of AI-driven underwater sensing technologies. The project has demonstrated significant improvements in underwater imaging quality and automatic underwater scene analysis, offering a robust framework for future advancements in oceanographic research, while opening the way for potential business exploitation of its results. Ultimately, iSEAu delivered advancements in underwater technology, towards improved marine research, industrial applications, and environmental monitoring.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

The iSEAu project pushes the boundaries of underwater imaging by developing a comprehensive suite of advanced techniques. It pioneers novel geometric correction methods, and significantly enhances radiometric correction through physics-aware deep learning and inverse problem formulations, extending to multispectral imaging for quantitative remote sensing. In Single-Photon Counting (SPC)-based imaging, iSEAu foresees backscatter removal and depth estimation, enabling accurate albedo recovery in extreme visibility conditions.

Furthermore, leveraging these advancements, iSEAu focuses on underwater scene analysis. It develops deep learning feature learning and 3D vision adaptations for robust geometric reconstruction, and adapts semantic segmentation methods with a focus on few-shot, weakly, and self-supervised learning to extract detailed semantic information like seabed composition and marine life monitoring. Finally, the project explores neural rendering for immersive 3D scene representation, incorporating multispectral data and addressing data scarcity, thereby advancing computer vision applications in underwater environments.

Multispectral 3D reconstruction of a hydrothermal vent chimney in Kolumbo

Restoration of underwater video frames based on the StreamUR method