Deep-Learning for Multimodal Sensor Fusion

Informations projet

DeeperSense

N° de convention de subvention: 101016958

DOI

10.3030/101016958

Projet clôturé

Date de signature de la CE 15 Decembre 2020

Date de début 1 Janvier 2021

Date de fin 31 Decembre 2023

Financé au titre de

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Coût total

€ 2 989 238,75

Contribution de l’UE

€ 2 989 238,75

Coordonné par

DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH
Germany

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Documents, reports (12)

ML Concept

Report on selection of baseline machine learning algorithms ANN topologies and concepts for modifications optimizations and algorithm training

Algorithm Testing & Validation Report

Report on the methodology applied and results of testing and algorithm validation.

Training Data Generation Report

Report on training data generation for the three use cases It includes description of data produced by the consortium either gathered in field campaigns or in simulation It is linked to the output of tasks T41 T42 and T43

Datasets for Public Dissemination

Report and description of datasets prepared for public dissemination in the Public Data Repository in WP7.

Evaluation ML Approaches

Report on indepth evaluation and selection of ML approaches available on basis of D23 and D24

Verification & Demonstration Concept

Report on concept for algorithm field validations usecase demonstrations and final joint field demonstration

Field Validation Report

Report and documentation of the field validations in Germany, Israel and Spain.

Algorithm Training & Optimization Results

Report on the modified core ML algorithms and the results of preliminary training 2nd iteration with data collected in WP4

Training Data Concept

Report on concepts for collection and generation of training data and algorithm testing taking into account D23

Sensor Concept

Report on feasible sensor pairings and specifications based on results of D21

Use-Case Requirements

Report describing the use cases and summarizing functional requirements for robotic systems for each use case

Final Demonstration Report

Report and documentation of final joint demonstration.

Open Research Data Pilot (2)

Public Data Repository

Public online repository for training data generated in DeeperSense, including metadata and documentation. PDR concept will be documented in D7.1.

Data Management Plan

This deliverable will formalize a data management plan according to the requirements of the Open Research Data Pilot

Other (3)

Communication & Dissemination Material

Website, social media channels, print material templates. Documentation of C&D Material will be included in D7.1.

Final DeeperSense Framework

Updated version of D3.3, with updated software and documentation.

Draft DeeperSense Framework

Preliminary version and documentation of software library with the (trained) core ML algorithms that will be applied to UC1, 2 and 3. Final version to be released as D5.3.

Publications

Conference proceedings (9)

Sonar-to-RGB Image Translation for Diver Monitoring in Poor Visibility Environments

Auteurs: Bilal Wehbe; Nimish Shah; Miguel Bande; Christian Backe
Publié dans: Oceans 2022, 2022
Éditeur: IEEE
DOI: 10.1109/oceans47191.2022.9977024

Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction

Auteurs: Arnold, Sascha; Wehbe, Bilal
Publié dans: IEEE International Conference on Robotics and Automation (ICRA) 2022, 2022
Éditeur: IEEE
DOI: 10.1109/icra46639.2022.9812277

Self-supervised Learning for Sonar Image Classification

Auteurs: Preciado-Grijalva, Alan; Wehbe, Bilal; Firvida, Miguel Bande; Valdenegro-Toro, Matias
Publié dans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, Page(s) 1499-1508
Éditeur: IEEE
DOI: 10.1109/cvprw56347.2022.00156

Pre-trained Models for Sonar Images

Auteurs: Valdenegro-Toro, Matias; Preciado-Grijalva, Alan; Wehbe, Bilal
Publié dans: Global Oceans 2021, 2021
Éditeur: arXiv
DOI: 10.23919/oceans44145.2021.9705825

Self-Supervised Monocular Depth Underwater

Auteurs: Amitai, Shlomi; Klein, Itzik; Treibitz, Tali
Publié dans: 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, Page(s) 1098-1104
Éditeur: IEEE
DOI: 10.1109/icra48891.2023.10161161

Creating Rich Metadata for Collaborative Research: Case Studies and Challenges

Auteurs: Backe, Christian; Gooran Orimi, Atefeh; Briken, Veit; Hamlaoui, Rayen; Görner, Hendrik
Publié dans: NFDI4Ing Conference 2023, Numéro 1, 2023
Éditeur: ZENODO
DOI: 10.5281/zenodo.8430752

Increasing diver safety for heavy underwater works by Sonar-to-Video Image Translation

Auteurs: J Lorscheidt, B Wehbe, D Cesar, T Vögele, T Becker
Publié dans: ISCRAM 2023, 2023
Éditeur: ISCRAM

Distortion Correction of AUV-acquired Side-Scan Sonar Data

Auteurs: V. Franchi, H. Rajani, R. Garcia, B. Martinez-Clavel and N. Gracias
Publié dans: OCEANS 2023 - Limerick, 2023, Page(s) 1-10
Éditeur: IEEE
DOI: 10.1109/oceanslimerick52467.2023.10244553

Enhancing the underwater vision to increase the safety of heavy underwater works by Intersensory learning – a use case in the European DeeperSense research project

Auteurs: Christian Illing, Tom Becker
Publié dans: Proceedings of MARESC 2021, 2021
Éditeur: NN

Thesis and dissertations (1)

FLSea: Underwater Visual-Inertial and Stereo-Vision Forward-Looking Datasets

Auteurs: Randall, Yelena; Treibitz, Tali
Publié dans: The International Journal of Robotics Research, Numéro Doctoral Thesis, 2023
Éditeur: University Haifa
DOI: 10.48550/arxiv.2302.12772

Peer reviewed articles (1)

A convolutional vision transformer for semantic segmentation of side-scan sonar data

Auteurs: Hayat Rajani; Nuno Gracias; Rafael Garcia
Publié dans: Ocean Engineering, Numéro 286, 2023, Page(s) 115647, ISSN 0029-8018
Éditeur: Pergamon Press Ltd.
DOI: 10.1016/j.oceaneng.2023.115647