Intelligent Scene Sensing and Analysis in Underwater Environments

Informazioni relative al progetto

iSEAu

ID dell’accordo di sovvenzione: 101030367

DOI

10.3030/101030367

Progetto chiuso

Data della firma CE 4 Maggio 2021

Data di avvio 1 Marzo 2022

Data di completamento 30 Dicembre 2024

Finanziato da

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Costo totale

€ 165 085,44

Contributo UE

€ 165 085,44

Coordinato da

ETHNICON METSOVION POLYTECHNION
Greece

Periodic Reporting for period 1 - iSEAu (Intelligent Scene Sensing and Analysis in Underwater Environments)

Periodo di rendicontazione: 2022-03-01 al 2024-02-29

Sintesi del contesto e degli obiettivi generali del progetto

Reliable and detailed underwater scene sensing, and analysis has a fundamental role in numerous underwater activities with scientific and economic interest. In fact, blue economy in 2020 amounted to €218 BN (GVA) and counts more than 5 million jobs in EU. At the same time, intensive research work focuses in marine and underwater environments including the study of climate change and its impacts, marine biology, underwater archaeology, marine geology, and marine renewable energy.

The main objective of iSEAu is to advance the scene sensing and perception capabilities in underwater environemnts by employing state-of-the-art machine learning methods, and using combinations of conventional (RGB) and non-conventional cameras, like multispectral and single photons cameras (SPCs).

iSEAu aims to bring together the fields of computer vision, machine learning and remote sensing for optimally addressing the underwater visual sensing challenges. The project objectives address this challenge in two
levels. The first concerns the development of methods for reducing the geometric and radiometric distortions introduced by the water through learning-based methods, and the development of methods based on transient imaging for perception in challenging visibility conditions. The second level concerns the adaptation and enhancement to the underwater domain of state-of-the-art methods for image-based extraction of structural and semantic information, and their field-testing considering representative application scenarios.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

The iSEAu project introduced significant advancements in underwater imaging and analysis through the development of advanced AI-driven methodologies. It successfully integrated deep learning, physics-based modelling, and specialized imaging techniques to enhance underwater image quality and enable intelligent scene understanding. Key achievements include the creation of StreamUR for real-time image restoration, MD2IP for training-free multispectral demosaicing, and a deep learning-powered system for autonomous gas bubble detection. Field deployments, notably in the deep hydrothermal field of Kolumbo underwater volcano near Santorini, Greece, and at the coastal hydrothermal field of Palaiochori in Milos, Greece, validated these methods, demonstrating their robustness in real-world environments. Furthermore, the project advanced low-power AI models for embedded systems, enabling their use in platforms with limited computational budget, as ROVs, AUVs and benthic stations. The work of the project led also to contributions in affine domains like satellite imaging and radiation mapping.

Overall, the results of the iSEAu project confirm the feasibility and effectiveness of AI-driven underwater sensing technologies. The project has demonstrated significant improvements in underwater imaging quality and automatic underwater scene analysis, offering a robust framework for future advancements in oceanographic research, while opening the way for potential business exploitation of its results. Ultimately, iSEAu delivered advancements in underwater technology, towards improved marine research, industrial applications, and environmental monitoring.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

The iSEAu project pushes the boundaries of underwater imaging by developing a comprehensive suite of advanced techniques. It pioneers novel geometric correction methods, and significantly enhances radiometric correction through physics-aware deep learning and inverse problem formulations, extending to multispectral imaging for quantitative remote sensing. In Single-Photon Counting (SPC)-based imaging, iSEAu foresees backscatter removal and depth estimation, enabling accurate albedo recovery in extreme visibility conditions.

Furthermore, leveraging these advancements, iSEAu focuses on underwater scene analysis. It develops deep learning feature learning and 3D vision adaptations for robust geometric reconstruction, and adapts semantic segmentation methods with a focus on few-shot, weakly, and self-supervised learning to extract detailed semantic information like seabed composition and marine life monitoring. Finally, the project explores neural rendering for immersive 3D scene representation, incorporating multispectral data and addressing data scarcity, thereby advancing computer vision applications in underwater environments.

Multispectral 3D reconstruction of a hydrothermal vent chimney in Kolumbo

Restoration of underwater video frames based on the StreamUR method