Deep-Learning for Multimodal Sensor Fusion

Informazioni relative al progetto

DeeperSense

ID dell’accordo di sovvenzione: 101016958

DOI

10.3030/101016958

Progetto chiuso

Data della firma CE 15 Dicembre 2020

Data di avvio 1 Gennaio 2021

Data di completamento 31 Dicembre 2023

Finanziato da

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Costo totale

€ 2 989 238,75

Contributo UE

€ 2 989 238,75

Coordinato da

DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH
Germany

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Risultati finali

Documents, reports (12)

ML Concept

Report on selection of baseline machine learning algorithms ANN topologies and concepts for modifications optimizations and algorithm training

Algorithm Testing & Validation Report

Report on the methodology applied and results of testing and algorithm validation.

Training Data Generation Report

Report on training data generation for the three use cases It includes description of data produced by the consortium either gathered in field campaigns or in simulation It is linked to the output of tasks T41 T42 and T43

Datasets for Public Dissemination

Report and description of datasets prepared for public dissemination in the Public Data Repository in WP7.

Evaluation ML Approaches

Report on indepth evaluation and selection of ML approaches available on basis of D23 and D24

Verification & Demonstration Concept

Report on concept for algorithm field validations usecase demonstrations and final joint field demonstration

Field Validation Report

Report and documentation of the field validations in Germany, Israel and Spain.

Algorithm Training & Optimization Results

Report on the modified core ML algorithms and the results of preliminary training 2nd iteration with data collected in WP4

Training Data Concept

Report on concepts for collection and generation of training data and algorithm testing taking into account D23

Sensor Concept

Report on feasible sensor pairings and specifications based on results of D21

Use-Case Requirements

Report describing the use cases and summarizing functional requirements for robotic systems for each use case

Final Demonstration Report

Report and documentation of final joint demonstration.

Open Research Data Pilot (2)

Public Data Repository

Public online repository for training data generated in DeeperSense, including metadata and documentation. PDR concept will be documented in D7.1.

Data Management Plan

This deliverable will formalize a data management plan according to the requirements of the Open Research Data Pilot

Other (3)

Communication & Dissemination Material

Website, social media channels, print material templates. Documentation of C&D Material will be included in D7.1.

Final DeeperSense Framework

Updated version of D3.3, with updated software and documentation.

Draft DeeperSense Framework

Preliminary version and documentation of software library with the (trained) core ML algorithms that will be applied to UC1, 2 and 3. Final version to be released as D5.3.

Pubblicazioni

Conference proceedings (9)

Sonar-to-RGB Image Translation for Diver Monitoring in Poor Visibility Environments

Autori: Bilal Wehbe; Nimish Shah; Miguel Bande; Christian Backe
Pubblicato in: Oceans 2022, 2022
Editore: IEEE
DOI: 10.1109/oceans47191.2022.9977024

Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction

Autori: Arnold, Sascha; Wehbe, Bilal
Pubblicato in: IEEE International Conference on Robotics and Automation (ICRA) 2022, 2022
Editore: IEEE
DOI: 10.1109/icra46639.2022.9812277

Self-supervised Learning for Sonar Image Classification

Autori: Preciado-Grijalva, Alan; Wehbe, Bilal; Firvida, Miguel Bande; Valdenegro-Toro, Matias
Pubblicato in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, Pagina/e 1499-1508
Editore: IEEE
DOI: 10.1109/cvprw56347.2022.00156

Pre-trained Models for Sonar Images

Autori: Valdenegro-Toro, Matias; Preciado-Grijalva, Alan; Wehbe, Bilal
Pubblicato in: Global Oceans 2021, 2021
Editore: arXiv
DOI: 10.23919/oceans44145.2021.9705825

Self-Supervised Monocular Depth Underwater

Autori: Amitai, Shlomi; Klein, Itzik; Treibitz, Tali
Pubblicato in: 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, Pagina/e 1098-1104
Editore: IEEE
DOI: 10.1109/icra48891.2023.10161161

Creating Rich Metadata for Collaborative Research: Case Studies and Challenges

Autori: Backe, Christian; Gooran Orimi, Atefeh; Briken, Veit; Hamlaoui, Rayen; Görner, Hendrik
Pubblicato in: NFDI4Ing Conference 2023, Numero 1, 2023
Editore: ZENODO
DOI: 10.5281/zenodo.8430752

Increasing diver safety for heavy underwater works by Sonar-to-Video Image Translation

Autori: J Lorscheidt, B Wehbe, D Cesar, T Vögele, T Becker
Pubblicato in: ISCRAM 2023, 2023
Editore: ISCRAM

Distortion Correction of AUV-acquired Side-Scan Sonar Data

Autori: V. Franchi, H. Rajani, R. Garcia, B. Martinez-Clavel and N. Gracias
Pubblicato in: OCEANS 2023 - Limerick, 2023, Pagina/e 1-10
Editore: IEEE
DOI: 10.1109/oceanslimerick52467.2023.10244553

Enhancing the underwater vision to increase the safety of heavy underwater works by Intersensory learning – a use case in the European DeeperSense research project

Autori: Christian Illing, Tom Becker
Pubblicato in: Proceedings of MARESC 2021, 2021
Editore: NN

Thesis and dissertations (1)

FLSea: Underwater Visual-Inertial and Stereo-Vision Forward-Looking Datasets

Autori: Randall, Yelena; Treibitz, Tali
Pubblicato in: The International Journal of Robotics Research, Numero Doctoral Thesis, 2023
Editore: University Haifa
DOI: 10.48550/arxiv.2302.12772

Peer reviewed articles (1)

A convolutional vision transformer for semantic segmentation of side-scan sonar data

Autori: Hayat Rajani; Nuno Gracias; Rafael Garcia
Pubblicato in: Ocean Engineering, Numero 286, 2023, Pagina/e 115647, ISSN 0029-8018
Editore: Pergamon Press Ltd.
DOI: 10.1016/j.oceaneng.2023.115647