Periodic Reporting for period 1 - MagicBathy (Multimodal multitAsk learninG for MultIsCale BATHYmetric mapping in shallow waters)
Période du rapport: 2023-02-01 au 2025-08-31
Coastal zones are vital socio-ecological systems, supporting biodiversity, economic activities, and the livelihoods of millions of people. Yet, they are increasingly threatened by climate change, coastal erosion, sea-level rise, and human-induced degradation. Accurate and up-to-date information on seabed topography (bathymetry) and seabed type is critical for effective coastal zone management, ecosystem monitoring, marine spatial planning, and climate adaptation strategies. However, such information remains scarce, expensive to acquire, and fragmented across space and time.
Traditional seabed mapping relies on in-situ surveys (e.g. echo-sounding, LiDAR), which are resource-intensive and geographically limited. Meanwhile, remote sensing (RS) imagery - from UAVs to satellites - offers broad spatial coverage, yet extracting accurate, high-resolution bathymetric and semantic information from it remains a significant scientific challenge. This is due to the complex physics of light propagation in water, diverse seabed characteristics, and the lack of robust, generalizable deep learning models tailored for underwater and shallow-water environments.
The MagicBathy project responds to these gaps by proposing a novel deep learning framework for joint bathymetry and seabed type mapping, uniquely designed for optical imagery in shallow coastal waters. It leverages recent advances in multitask learning, in-domain representation learning, and super-resolution, aiming to transform how bathymetric and semantic information is derived from remote sensing data.
The overarching objectives of the project were to:
- Enable fine-grained, simultaneous prediction of depth and seabed type by developing multitask models that reflect the interdependent nature of the two outputs.
- Establish domain-specific visual representations that account for seabed texture, water attenuation, and modality-induced variations in RS imagery.
- Boost the resolution and usability of satellite and aerial data through a novel super-resolution framework informed by UAV-based data and contextual scene knowledge.
Given its relevance to marine environmental governance, EU coastal policy, and the Digital Twin of the Ocean (DTO) initiative, MagicBathy's reults are expected to have a significant impact on both scientific advancement and societal needs. They will support data-driven decision-making, reduce monitoring costs, and democratize access to critical coastal information across Europe and beyond.
Traditional seabed mapping relies on in-situ surveys (e.g. echo-sounding, LiDAR), which are resource-intensive and geographically limited. Meanwhile, remote sensing (RS) imagery - from UAVs to satellites - offers broad spatial coverage, yet extracting accurate, high-resolution bathymetric and semantic information from it remains a significant scientific challenge. This is due to the complex physics of light propagation in water, diverse seabed characteristics, and the lack of robust, generalizable deep learning models tailored for underwater and shallow-water environments.
The MagicBathy project responds to these gaps by proposing a novel deep learning framework for joint bathymetry and seabed type mapping, uniquely designed for optical imagery in shallow coastal waters. It leverages recent advances in multitask learning, in-domain representation learning, and super-resolution, aiming to transform how bathymetric and semantic information is derived from remote sensing data.
The overarching objectives of the project were to:
- Enable fine-grained, simultaneous prediction of depth and seabed type by developing multitask models that reflect the interdependent nature of the two outputs.
- Establish domain-specific visual representations that account for seabed texture, water attenuation, and modality-induced variations in RS imagery.
- Boost the resolution and usability of satellite and aerial data through a novel super-resolution framework informed by UAV-based data and contextual scene knowledge.
Given its relevance to marine environmental governance, EU coastal policy, and the Digital Twin of the Ocean (DTO) initiative, MagicBathy's reults are expected to have a significant impact on both scientific advancement and societal needs. They will support data-driven decision-making, reduce monitoring costs, and democratize access to critical coastal information across Europe and beyond.
The MagicBathy project has delivered substantial scientific and technical progress toward enabling accurate, scalable, and multimodal shallow-water mapping through deep learning and remote sensing. The following key research activities and outcomes were achieved:
1. MagicBathyNet Dataset Development
A cornerstone contribution of the project is the creation of MagicBathyNet, the first publicly available multimodal benchmark dataset (available at https://zenodo.org/records/10470959(s’ouvre dans une nouvelle fenêtre)) for joint bathymetry and seabed classification from optical imagery. The dataset consists of co-registered image patches from Sentinel-2, SPOT-6, and aerial platforms, along with bathymetry in raster format and corresponding pixel-level seabed class annotations. This enables supervised training and benchmarking of deep models under realistic multisensor scenarios.
In addition to labeled data, MagicBathyNet includes a large volume of unlabeled samples, facilitating self-supervised pretraining strategies prior to task-specific fine-tuning. This makes it a uniquely valuable resource for advancing representation learning in underwater and coastal remote sensing. The dataset has been curated across diverse coastal regions, capturing environmental variability and ensuring generalizability of trained models.
MagicBathyNet was exploited to benchmark state-of-the-art methods in learning-based bathymetry and pixel-based classification.
2. Sea-Undistort Synthetic Dataset
To overcome the lack of real-world paired data for evaluating and correcting through-water image distortion, we developed Sea-Undistort (available at https://zenodo.org/records/15639838(s’ouvre dans une nouvelle fenêtre)) a synthetic dataset comprising 1200 high-resolution (512×512) image pairs rendered in Blender. Each pair includes a distortion-free and a distorted underwater view, with realistic effects such as sun glint, water surface waves, and scattering over a variety of seabed types.
Crucially, each image is accompanied by metadata such as camera parameters, sun position, and average depth, enabling highly controlled, supervised training of distortion-correcting models, including both generative and non-generative architectures. Sea-Undistort is the first of its kind, paving the way for foundational research in underwater image correction and self-supervised learning in domains where clean references are not physically attainable.
We used Sea-Undistort to benchmark state-of-the-art image restoration methods alongside our proposed variant; an enhanced lightweight diffusion-based framework with an early-fusion sun-glint mask. When applied to real aerial data, the proposed variant delivers more complete Digital Surface Models (DSMs) of the seabed, especially in deeper areas, reduces bathymetric errors, suppresses glint and scattering, and crisply restores fine seabed details.
3. Swin-BathyUNet architecture for Spectrally Derived Bathymetry
To address limitations in both Spectrally Derived Bathymetry (SDB) and SfM-MVS approaches - such as missing depth data in texture-poor regions - we proposed Swin-BathyUNet, a novel architecture that fuses U-Net with Swin Transformer-based self-attention and cross-attention mechanisms.
It can:
• Operate as a standalone SDB method, or
• Leverage incomplete SfM-MVS DSMs to learn complete bathymetric predictions.
The model captures long-range spatial dependencies and integrates spectral and geometric cues. Experiments in diverse test sites (Mediterranean and Baltic Seas) demonstrate that Swin-BathyUNet significantly improves bathymetric accuracy, completeness, and noise robustness over state-of-the-art methods, producing more reliable Digital Surface Models (DSMs) in challenging shallow-water environments.
4. Seabed-Net architecture: Joint Bathymetry and Seabed Classification
Recognizing the limitations of treating depth estimation and seabed classification as separate tasks, we introduced Seabed-Net, a unified multitask architecture that predicts both bathymetry and seabed class maps from multispectral remote sensing imagery.
Key features include:
• Dual-task encoder-decoder branches,
• An Attention Feature Fusion (AFF) module,
• A windowed Swin Transformer for multi-scale feature integration, and
• Dynamic task uncertainty weighting to balance learning across tasks.
Seabed-Net consistently outperforms single-task and multitask baselines, achieving 10–30% lower RMSE in depth estimation and up to 8% higher accuracy in seabed classification. The model enhances spatial coherence, improves detection in low-contrast substrates, and strengthens semantic boundary localization. These results confirm that joint modeling of bathymetry and substrate improves both tasks and enables richer environmental characterization.
5. Large-scale pretraining for Ocean Remote Sensing
The project explored the development of foundational models for ocean remote sensing by critically evaluating various self-supervised learning (SSL) paradigms, particularly:
• Contrastive vs. generative SSL,
• The integration of geolocation and water quality metadata, and
• Transfer performance on tasks like bathymetry estimation and marine debris detection.
This research contributes to a better understanding of how large-scale, unlabeled remote sensing datasets can be harnessed to pretrain general-purpose models capable of efficient fine-tuning on downstream ocean monitoring tasks.
6. Super-Resolution for Optical Imagery in Coastal Mapping
Lastly, research was conducted into super-resolution methods for enhancing the spatial detail of satellite and aerial imagery used in seabed mapping.
Both bathymetry-guided and agnostic variants were explored to evaluate their capacity to:
• Recover fine-scale seabed structure,
• Improve feature separation in mixed substrates, and
• Enable high-resolution predictions from low-resolution inputs (e.g. Sentinel-2).
Preliminary results show promising gains in restoring edge sharpness and improving model performance in downstream bathymetry and classification tasks.
1. MagicBathyNet Dataset Development
A cornerstone contribution of the project is the creation of MagicBathyNet, the first publicly available multimodal benchmark dataset (available at https://zenodo.org/records/10470959(s’ouvre dans une nouvelle fenêtre)) for joint bathymetry and seabed classification from optical imagery. The dataset consists of co-registered image patches from Sentinel-2, SPOT-6, and aerial platforms, along with bathymetry in raster format and corresponding pixel-level seabed class annotations. This enables supervised training and benchmarking of deep models under realistic multisensor scenarios.
In addition to labeled data, MagicBathyNet includes a large volume of unlabeled samples, facilitating self-supervised pretraining strategies prior to task-specific fine-tuning. This makes it a uniquely valuable resource for advancing representation learning in underwater and coastal remote sensing. The dataset has been curated across diverse coastal regions, capturing environmental variability and ensuring generalizability of trained models.
MagicBathyNet was exploited to benchmark state-of-the-art methods in learning-based bathymetry and pixel-based classification.
2. Sea-Undistort Synthetic Dataset
To overcome the lack of real-world paired data for evaluating and correcting through-water image distortion, we developed Sea-Undistort (available at https://zenodo.org/records/15639838(s’ouvre dans une nouvelle fenêtre)) a synthetic dataset comprising 1200 high-resolution (512×512) image pairs rendered in Blender. Each pair includes a distortion-free and a distorted underwater view, with realistic effects such as sun glint, water surface waves, and scattering over a variety of seabed types.
Crucially, each image is accompanied by metadata such as camera parameters, sun position, and average depth, enabling highly controlled, supervised training of distortion-correcting models, including both generative and non-generative architectures. Sea-Undistort is the first of its kind, paving the way for foundational research in underwater image correction and self-supervised learning in domains where clean references are not physically attainable.
We used Sea-Undistort to benchmark state-of-the-art image restoration methods alongside our proposed variant; an enhanced lightweight diffusion-based framework with an early-fusion sun-glint mask. When applied to real aerial data, the proposed variant delivers more complete Digital Surface Models (DSMs) of the seabed, especially in deeper areas, reduces bathymetric errors, suppresses glint and scattering, and crisply restores fine seabed details.
3. Swin-BathyUNet architecture for Spectrally Derived Bathymetry
To address limitations in both Spectrally Derived Bathymetry (SDB) and SfM-MVS approaches - such as missing depth data in texture-poor regions - we proposed Swin-BathyUNet, a novel architecture that fuses U-Net with Swin Transformer-based self-attention and cross-attention mechanisms.
It can:
• Operate as a standalone SDB method, or
• Leverage incomplete SfM-MVS DSMs to learn complete bathymetric predictions.
The model captures long-range spatial dependencies and integrates spectral and geometric cues. Experiments in diverse test sites (Mediterranean and Baltic Seas) demonstrate that Swin-BathyUNet significantly improves bathymetric accuracy, completeness, and noise robustness over state-of-the-art methods, producing more reliable Digital Surface Models (DSMs) in challenging shallow-water environments.
4. Seabed-Net architecture: Joint Bathymetry and Seabed Classification
Recognizing the limitations of treating depth estimation and seabed classification as separate tasks, we introduced Seabed-Net, a unified multitask architecture that predicts both bathymetry and seabed class maps from multispectral remote sensing imagery.
Key features include:
• Dual-task encoder-decoder branches,
• An Attention Feature Fusion (AFF) module,
• A windowed Swin Transformer for multi-scale feature integration, and
• Dynamic task uncertainty weighting to balance learning across tasks.
Seabed-Net consistently outperforms single-task and multitask baselines, achieving 10–30% lower RMSE in depth estimation and up to 8% higher accuracy in seabed classification. The model enhances spatial coherence, improves detection in low-contrast substrates, and strengthens semantic boundary localization. These results confirm that joint modeling of bathymetry and substrate improves both tasks and enables richer environmental characterization.
5. Large-scale pretraining for Ocean Remote Sensing
The project explored the development of foundational models for ocean remote sensing by critically evaluating various self-supervised learning (SSL) paradigms, particularly:
• Contrastive vs. generative SSL,
• The integration of geolocation and water quality metadata, and
• Transfer performance on tasks like bathymetry estimation and marine debris detection.
This research contributes to a better understanding of how large-scale, unlabeled remote sensing datasets can be harnessed to pretrain general-purpose models capable of efficient fine-tuning on downstream ocean monitoring tasks.
6. Super-Resolution for Optical Imagery in Coastal Mapping
Lastly, research was conducted into super-resolution methods for enhancing the spatial detail of satellite and aerial imagery used in seabed mapping.
Both bathymetry-guided and agnostic variants were explored to evaluate their capacity to:
• Recover fine-scale seabed structure,
• Improve feature separation in mixed substrates, and
• Enable high-resolution predictions from low-resolution inputs (e.g. Sentinel-2).
Preliminary results show promising gains in restoring edge sharpness and improving model performance in downstream bathymetry and classification tasks.
The MagicBathy project has successfully delivered a series of scientific and technical advancements aimed at revolutionizing shallow-water seabed mapping through deep learning, multisensor data fusion, and synthetic data generation. The following summarizes the major outcomes and their potential impacts, along with key considerations for further uptake and long-term success.
Overview of Results
• MagicBathyNet Dataset: The first open-access, multimodal benchmark dataset combining satellite, aerial imagery, bathymetry, and seabed classification annotations. It establishes a critical resource for benchmarking models and pretraining foundation models for ocean remote sensing.
• Sea-Undistort Synthetic Dataset: A novel synthetic dataset of underwater image pairs with realistic distortions, sun glint, and scattering, accompanied by full metadata. It enables supervised training of image correction models where real-world paired data are unattainable.
• Swin-BathyUNet: A hybrid SDB deep learning model combining the geometric strength of SfM-MVS with spectral input and attention-based learning. It outperforms existing SDB methods in terms of depth accuracy, noise reduction, and coverage across challenging environments.
• Seabed-Net: A multitask model for simultaneous bathymetry and seabed classification, improving prediction accuracy for both tasks. It outperforms existing multitask models. It delivers better generalization, enhanced spatial consistency, and supports integrated habitat mapping for environmental monitoring.
• Research on Self-Supervised Representation Learning: Evaluation of contrastive vs. generative pretraining paradigms, and incorporation of geolocation priors. Findings support the design of large-scale, reusable models for bathymetry and marine debris detection.
• Research on Super-Resolution: Preliminary models show the ability to enhance spatial fidelity of low-resolution imagery, enabling more detailed seabed analysis using coarser satellite data.
Potential Impacts
1. Scientific Advancement
◦ Lays the foundation for next-generation remote sensing models in ocean and coastal environments, especially in under-mapped and visually challenging areas.
◦ Enables reproducible research and benchmarking through open-access datasets and code.
◦ Facilitates progress toward automated, high-frequency, and large-scale shallow-water monitoring.
2. Environmental and Societal Relevance
◦ Supports coastal zone management, marine habitat conservation, and monitoring of areas affected by climate change or anthropogenic pressure.
◦ Contributes to global initiatives such as Seabed 2030, the UN Ocean Decade, and the EU Biodiversity Strategy.
3. Technological and Industrial Innovation
◦ Demonstrates potential for commercial downstream services, including shallow-water charting, habitat surveillance, and autonomous vehicle navigation.
◦ Opens new directions for hybrid solutions combining physical modeling (e.g. SfM-MVS) with learning-based methods.
Key Needs to Ensure Further Uptake and Success
To maximize the uptake and exploitation of the results, the following areas have been identified as critical:
1. Further Research & Demonstration
◦ Larger-scale experiments with operational coastal authorities, NGOs, and mapping agencies to validate model robustness under varied conditions.
◦ Extension of Seabed-Net and Swin-BathyUNet to temporal analysis, enabling monitoring of seabed evolution.
2. Access to Markets and Commercialisation
◦ Partnerships with marine data providers, ocean tech SMEs, and earth observation startups are needed to co-develop practical applications.
◦ Development of API-based services for bathymetry and seabed classification from imagery platforms.
3. IPR and Open Access
◦ Key models and datasets are published under permissive open-source licenses, while allowing for downstream IP protection of derivative services.
4. Standardisation and Regulatory Alignment
◦ Engagement with hydrographic offices, INSPIRE-compliant data infrastructures, and marine spatial planning frameworks to ensure standards compliance.
◦ Alignment with Copernicus marine service protocols for wider integration into European Earth Observation services.
5. Internationalisation
◦ Dissemination through international conferences and working groups (e.g. GEBCO, IOC/UNESCO).
◦ Exploration of international collaborations to expand MagicBathyNet to tropical and arctic regions, increasing global coverage.
In summary, the MagicBathy project has produced novel datasets, algorithms, and workflows that address core limitations of existing seabed mapping approaches. With strategic follow-up actions in research, standardisation, and commercialisation, these results hold strong potential to become foundational components of future coastal and marine monitoring pipelines.
Overview of Results
• MagicBathyNet Dataset: The first open-access, multimodal benchmark dataset combining satellite, aerial imagery, bathymetry, and seabed classification annotations. It establishes a critical resource for benchmarking models and pretraining foundation models for ocean remote sensing.
• Sea-Undistort Synthetic Dataset: A novel synthetic dataset of underwater image pairs with realistic distortions, sun glint, and scattering, accompanied by full metadata. It enables supervised training of image correction models where real-world paired data are unattainable.
• Swin-BathyUNet: A hybrid SDB deep learning model combining the geometric strength of SfM-MVS with spectral input and attention-based learning. It outperforms existing SDB methods in terms of depth accuracy, noise reduction, and coverage across challenging environments.
• Seabed-Net: A multitask model for simultaneous bathymetry and seabed classification, improving prediction accuracy for both tasks. It outperforms existing multitask models. It delivers better generalization, enhanced spatial consistency, and supports integrated habitat mapping for environmental monitoring.
• Research on Self-Supervised Representation Learning: Evaluation of contrastive vs. generative pretraining paradigms, and incorporation of geolocation priors. Findings support the design of large-scale, reusable models for bathymetry and marine debris detection.
• Research on Super-Resolution: Preliminary models show the ability to enhance spatial fidelity of low-resolution imagery, enabling more detailed seabed analysis using coarser satellite data.
Potential Impacts
1. Scientific Advancement
◦ Lays the foundation for next-generation remote sensing models in ocean and coastal environments, especially in under-mapped and visually challenging areas.
◦ Enables reproducible research and benchmarking through open-access datasets and code.
◦ Facilitates progress toward automated, high-frequency, and large-scale shallow-water monitoring.
2. Environmental and Societal Relevance
◦ Supports coastal zone management, marine habitat conservation, and monitoring of areas affected by climate change or anthropogenic pressure.
◦ Contributes to global initiatives such as Seabed 2030, the UN Ocean Decade, and the EU Biodiversity Strategy.
3. Technological and Industrial Innovation
◦ Demonstrates potential for commercial downstream services, including shallow-water charting, habitat surveillance, and autonomous vehicle navigation.
◦ Opens new directions for hybrid solutions combining physical modeling (e.g. SfM-MVS) with learning-based methods.
Key Needs to Ensure Further Uptake and Success
To maximize the uptake and exploitation of the results, the following areas have been identified as critical:
1. Further Research & Demonstration
◦ Larger-scale experiments with operational coastal authorities, NGOs, and mapping agencies to validate model robustness under varied conditions.
◦ Extension of Seabed-Net and Swin-BathyUNet to temporal analysis, enabling monitoring of seabed evolution.
2. Access to Markets and Commercialisation
◦ Partnerships with marine data providers, ocean tech SMEs, and earth observation startups are needed to co-develop practical applications.
◦ Development of API-based services for bathymetry and seabed classification from imagery platforms.
3. IPR and Open Access
◦ Key models and datasets are published under permissive open-source licenses, while allowing for downstream IP protection of derivative services.
4. Standardisation and Regulatory Alignment
◦ Engagement with hydrographic offices, INSPIRE-compliant data infrastructures, and marine spatial planning frameworks to ensure standards compliance.
◦ Alignment with Copernicus marine service protocols for wider integration into European Earth Observation services.
5. Internationalisation
◦ Dissemination through international conferences and working groups (e.g. GEBCO, IOC/UNESCO).
◦ Exploration of international collaborations to expand MagicBathyNet to tropical and arctic regions, increasing global coverage.
In summary, the MagicBathy project has produced novel datasets, algorithms, and workflows that address core limitations of existing seabed mapping approaches. With strategic follow-up actions in research, standardisation, and commercialisation, these results hold strong potential to become foundational components of future coastal and marine monitoring pipelines.