Intelligent Scene Sensing and Analysis in Underwater Environments

Información del proyecto

iSEAu

Identificador del acuerdo de subvención: 101030367

DOI

10.3030/101030367

Proyecto cerrado

Fecha de la firma de la CE 4 Mayo 2021

Fecha de inicio 1 Marzo 2022

Fecha de finalización 30 Diciembre 2024

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 165 085,44

Aportación de la UE

€ 165 085,44

Coordinado por

ETHNICON METSOVION POLYTECHNION
Greece

Periodic Reporting for period 1 - iSEAu (Intelligent Scene Sensing and Analysis in Underwater Environments)

Período documentado: 2022-03-01 hasta 2024-02-29

Resumen del contexto y de los objetivos generales del proyecto

Reliable and detailed underwater scene sensing, and analysis has a fundamental role in numerous underwater activities with scientific and economic interest. In fact, blue economy in 2020 amounted to €218 BN (GVA) and counts more than 5 million jobs in EU. At the same time, intensive research work focuses in marine and underwater environments including the study of climate change and its impacts, marine biology, underwater archaeology, marine geology, and marine renewable energy.

The main objective of iSEAu is to advance the scene sensing and perception capabilities in underwater environemnts by employing state-of-the-art machine learning methods, and using combinations of conventional (RGB) and non-conventional cameras, like multispectral and single photons cameras (SPCs).

iSEAu aims to bring together the fields of computer vision, machine learning and remote sensing for optimally addressing the underwater visual sensing challenges. The project objectives address this challenge in two
levels. The first concerns the development of methods for reducing the geometric and radiometric distortions introduced by the water through learning-based methods, and the development of methods based on transient imaging for perception in challenging visibility conditions. The second level concerns the adaptation and enhancement to the underwater domain of state-of-the-art methods for image-based extraction of structural and semantic information, and their field-testing considering representative application scenarios.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

The iSEAu project introduced significant advancements in underwater imaging and analysis through the development of advanced AI-driven methodologies. It successfully integrated deep learning, physics-based modelling, and specialized imaging techniques to enhance underwater image quality and enable intelligent scene understanding. Key achievements include the creation of StreamUR for real-time image restoration, MD2IP for training-free multispectral demosaicing, and a deep learning-powered system for autonomous gas bubble detection. Field deployments, notably in the deep hydrothermal field of Kolumbo underwater volcano near Santorini, Greece, and at the coastal hydrothermal field of Palaiochori in Milos, Greece, validated these methods, demonstrating their robustness in real-world environments. Furthermore, the project advanced low-power AI models for embedded systems, enabling their use in platforms with limited computational budget, as ROVs, AUVs and benthic stations. The work of the project led also to contributions in affine domains like satellite imaging and radiation mapping.

Overall, the results of the iSEAu project confirm the feasibility and effectiveness of AI-driven underwater sensing technologies. The project has demonstrated significant improvements in underwater imaging quality and automatic underwater scene analysis, offering a robust framework for future advancements in oceanographic research, while opening the way for potential business exploitation of its results. Ultimately, iSEAu delivered advancements in underwater technology, towards improved marine research, industrial applications, and environmental monitoring.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

The iSEAu project pushes the boundaries of underwater imaging by developing a comprehensive suite of advanced techniques. It pioneers novel geometric correction methods, and significantly enhances radiometric correction through physics-aware deep learning and inverse problem formulations, extending to multispectral imaging for quantitative remote sensing. In Single-Photon Counting (SPC)-based imaging, iSEAu foresees backscatter removal and depth estimation, enabling accurate albedo recovery in extreme visibility conditions.

Furthermore, leveraging these advancements, iSEAu focuses on underwater scene analysis. It develops deep learning feature learning and 3D vision adaptations for robust geometric reconstruction, and adapts semantic segmentation methods with a focus on few-shot, weakly, and self-supervised learning to extract detailed semantic information like seabed composition and marine life monitoring. Finally, the project explores neural rendering for immersive 3D scene representation, incorporating multispectral data and addressing data scarcity, thereby advancing computer vision applications in underwater environments.

Multispectral 3D reconstruction of a hydrothermal vent chimney in Kolumbo

Restoration of underwater video frames based on the StreamUR method