Deep-Learning for Multimodal Sensor Fusion

Projektinformationen

DeeperSense

ID Finanzhilfevereinbarung: 101016958

DOI

10.3030/101016958

Projekt abgeschlossen

EK-Unterschriftsdatum 15 Dezember 2020

Startdatum 1 Januar 2021

Enddatum 31 Dezember 2023

Finanziert unter

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Gesamtkosten

€ 2 989 238,75

EU-Beitrag

€ 2 989 238,75

Koordiniert durch

DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH
Germany

CORDIS bietet Links zu öffentlichen Ergebnissen und Veröffentlichungen von HORIZONT-Projekten.

Links zu Ergebnissen und Veröffentlichungen von RP7-Projekten sowie Links zu einigen Typen spezifischer Ergebnisse wie Datensätzen und Software werden dynamisch von OpenAIRE abgerufen.

Leistungen

Documents, reports (12)

ML Concept

Report on selection of baseline machine learning algorithms ANN topologies and concepts for modifications optimizations and algorithm training

Algorithm Testing & Validation Report

Report on the methodology applied and results of testing and algorithm validation.

Training Data Generation Report

Report on training data generation for the three use cases It includes description of data produced by the consortium either gathered in field campaigns or in simulation It is linked to the output of tasks T41 T42 and T43

Datasets for Public Dissemination

Report and description of datasets prepared for public dissemination in the Public Data Repository in WP7.

Evaluation ML Approaches

Report on indepth evaluation and selection of ML approaches available on basis of D23 and D24

Verification & Demonstration Concept

Report on concept for algorithm field validations usecase demonstrations and final joint field demonstration

Field Validation Report

Report and documentation of the field validations in Germany, Israel and Spain.

Algorithm Training & Optimization Results

Report on the modified core ML algorithms and the results of preliminary training 2nd iteration with data collected in WP4

Training Data Concept

Report on concepts for collection and generation of training data and algorithm testing taking into account D23

Sensor Concept

Report on feasible sensor pairings and specifications based on results of D21

Use-Case Requirements

Report describing the use cases and summarizing functional requirements for robotic systems for each use case

Final Demonstration Report

Report and documentation of final joint demonstration.

Open Research Data Pilot (2)

Public Data Repository

Public online repository for training data generated in DeeperSense, including metadata and documentation. PDR concept will be documented in D7.1.

Data Management Plan

This deliverable will formalize a data management plan according to the requirements of the Open Research Data Pilot

Other (3)

Communication & Dissemination Material

Website, social media channels, print material templates. Documentation of C&D Material will be included in D7.1.

Final DeeperSense Framework

Updated version of D3.3, with updated software and documentation.

Draft DeeperSense Framework

Preliminary version and documentation of software library with the (trained) core ML algorithms that will be applied to UC1, 2 and 3. Final version to be released as D5.3.

Veröffentlichungen

Conference proceedings (9)

Sonar-to-RGB Image Translation for Diver Monitoring in Poor Visibility Environments

Autoren: Bilal Wehbe; Nimish Shah; Miguel Bande; Christian Backe
Veröffentlicht in: Oceans 2022, 2022
Herausgeber: IEEE
DOI: 10.1109/oceans47191.2022.9977024

Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction

Autoren: Arnold, Sascha; Wehbe, Bilal
Veröffentlicht in: IEEE International Conference on Robotics and Automation (ICRA) 2022, 2022
Herausgeber: IEEE
DOI: 10.1109/icra46639.2022.9812277

Self-supervised Learning for Sonar Image Classification

Autoren: Preciado-Grijalva, Alan; Wehbe, Bilal; Firvida, Miguel Bande; Valdenegro-Toro, Matias
Veröffentlicht in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, Seite(n) 1499-1508
Herausgeber: IEEE
DOI: 10.1109/cvprw56347.2022.00156

Pre-trained Models for Sonar Images

Autoren: Valdenegro-Toro, Matias; Preciado-Grijalva, Alan; Wehbe, Bilal
Veröffentlicht in: Global Oceans 2021, 2021
Herausgeber: arXiv
DOI: 10.23919/oceans44145.2021.9705825

Self-Supervised Monocular Depth Underwater

Autoren: Amitai, Shlomi; Klein, Itzik; Treibitz, Tali
Veröffentlicht in: 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, Seite(n) 1098-1104
Herausgeber: IEEE
DOI: 10.1109/icra48891.2023.10161161

Creating Rich Metadata for Collaborative Research: Case Studies and Challenges

Autoren: Backe, Christian; Gooran Orimi, Atefeh; Briken, Veit; Hamlaoui, Rayen; Görner, Hendrik
Veröffentlicht in: NFDI4Ing Conference 2023, Ausgabe 1, 2023
Herausgeber: ZENODO
DOI: 10.5281/zenodo.8430752

Increasing diver safety for heavy underwater works by Sonar-to-Video Image Translation

Autoren: J Lorscheidt, B Wehbe, D Cesar, T Vögele, T Becker
Veröffentlicht in: ISCRAM 2023, 2023
Herausgeber: ISCRAM

Distortion Correction of AUV-acquired Side-Scan Sonar Data

Autoren: V. Franchi, H. Rajani, R. Garcia, B. Martinez-Clavel and N. Gracias
Veröffentlicht in: OCEANS 2023 - Limerick, 2023, Seite(n) 1-10
Herausgeber: IEEE
DOI: 10.1109/oceanslimerick52467.2023.10244553

Enhancing the underwater vision to increase the safety of heavy underwater works by Intersensory learning – a use case in the European DeeperSense research project

Autoren: Christian Illing, Tom Becker
Veröffentlicht in: Proceedings of MARESC 2021, 2021
Herausgeber: NN

Thesis and dissertations (1)

FLSea: Underwater Visual-Inertial and Stereo-Vision Forward-Looking Datasets

Autoren: Randall, Yelena; Treibitz, Tali
Veröffentlicht in: The International Journal of Robotics Research, Ausgabe Doctoral Thesis, 2023
Herausgeber: University Haifa
DOI: 10.48550/arxiv.2302.12772

Peer reviewed articles (1)

A convolutional vision transformer for semantic segmentation of side-scan sonar data

Autoren: Hayat Rajani; Nuno Gracias; Rafael Garcia
Veröffentlicht in: Ocean Engineering, Ausgabe 286, 2023, Seite(n) 115647, ISSN 0029-8018
Herausgeber: Pergamon Press Ltd.
DOI: 10.1016/j.oceaneng.2023.115647