Intelligent Scene Sensing and Analysis in Underwater Environments

Projektinformationen

iSEAu

ID Finanzhilfevereinbarung: 101030367

DOI

10.3030/101030367

Projekt abgeschlossen

EK-Unterschriftsdatum 4 Mai 2021

Startdatum 1 März 2022

Enddatum 30 Dezember 2024

Finanziert unter

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Gesamtkosten

€ 165 085,44

EU-Beitrag

€ 165 085,44

Koordiniert durch

ETHNICON METSOVION POLYTECHNION
Greece

Periodic Reporting for period 1 - iSEAu (Intelligent Scene Sensing and Analysis in Underwater Environments)

Berichtszeitraum: 2022-03-01 bis 2024-02-29

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

Reliable and detailed underwater scene sensing, and analysis has a fundamental role in numerous underwater activities with scientific and economic interest. In fact, blue economy in 2020 amounted to €218 BN (GVA) and counts more than 5 million jobs in EU. At the same time, intensive research work focuses in marine and underwater environments including the study of climate change and its impacts, marine biology, underwater archaeology, marine geology, and marine renewable energy.

The main objective of iSEAu is to advance the scene sensing and perception capabilities in underwater environemnts by employing state-of-the-art machine learning methods, and using combinations of conventional (RGB) and non-conventional cameras, like multispectral and single photons cameras (SPCs).

iSEAu aims to bring together the fields of computer vision, machine learning and remote sensing for optimally addressing the underwater visual sensing challenges. The project objectives address this challenge in two
levels. The first concerns the development of methods for reducing the geometric and radiometric distortions introduced by the water through learning-based methods, and the development of methods based on transient imaging for perception in challenging visibility conditions. The second level concerns the adaptation and enhancement to the underwater domain of state-of-the-art methods for image-based extraction of structural and semantic information, and their field-testing considering representative application scenarios.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

The iSEAu project introduced significant advancements in underwater imaging and analysis through the development of advanced AI-driven methodologies. It successfully integrated deep learning, physics-based modelling, and specialized imaging techniques to enhance underwater image quality and enable intelligent scene understanding. Key achievements include the creation of StreamUR for real-time image restoration, MD2IP for training-free multispectral demosaicing, and a deep learning-powered system for autonomous gas bubble detection. Field deployments, notably in the deep hydrothermal field of Kolumbo underwater volcano near Santorini, Greece, and at the coastal hydrothermal field of Palaiochori in Milos, Greece, validated these methods, demonstrating their robustness in real-world environments. Furthermore, the project advanced low-power AI models for embedded systems, enabling their use in platforms with limited computational budget, as ROVs, AUVs and benthic stations. The work of the project led also to contributions in affine domains like satellite imaging and radiation mapping.

Overall, the results of the iSEAu project confirm the feasibility and effectiveness of AI-driven underwater sensing technologies. The project has demonstrated significant improvements in underwater imaging quality and automatic underwater scene analysis, offering a robust framework for future advancements in oceanographic research, while opening the way for potential business exploitation of its results. Ultimately, iSEAu delivered advancements in underwater technology, towards improved marine research, industrial applications, and environmental monitoring.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

The iSEAu project pushes the boundaries of underwater imaging by developing a comprehensive suite of advanced techniques. It pioneers novel geometric correction methods, and significantly enhances radiometric correction through physics-aware deep learning and inverse problem formulations, extending to multispectral imaging for quantitative remote sensing. In Single-Photon Counting (SPC)-based imaging, iSEAu foresees backscatter removal and depth estimation, enabling accurate albedo recovery in extreme visibility conditions.

Furthermore, leveraging these advancements, iSEAu focuses on underwater scene analysis. It develops deep learning feature learning and 3D vision adaptations for robust geometric reconstruction, and adapts semantic segmentation methods with a focus on few-shot, weakly, and self-supervised learning to extract detailed semantic information like seabed composition and marine life monitoring. Finally, the project explores neural rendering for immersive 3D scene representation, incorporating multispectral data and addressing data scarcity, thereby advancing computer vision applications in underwater environments.

Multispectral 3D reconstruction of a hydrothermal vent chimney in Kolumbo

Restoration of underwater video frames based on the StreamUR method