Deep-Learning for Multimodal Sensor Fusion

Objective

The main objective of DeeperSense is to significantly improve the capabilities for environment perception of service robots to improve their performance and reliability, achieve new functionality, and open up new applications for robotics. DeeperSense adopts a novel approach of using Artificial Intelligence and data-driven Machine Learning / DeepLearning to combine the capabilities of non-visual and visual sensors with the objective to improve their joint capability of environment perception beyond the capabilities of the individual sensors. As one of the most challenging application areas for robot operation and environment perception, DeeperSense chooses underwater robotics as a domain to demonstrate and verify this approach. The project implements DeepLearning solutions for three use cases that were selected for their societal relevance and are driven by concrete end-user and market needs. During the project, comprehensive training data are generated. The algorithms are trained on these data and verified both in the lab and in extensive field trials. The trained algorithms are optimized to run on the on-board hardware of underwater vehicles, thus enabling real-time execution in support of the autonomous robot behaviour. Both the algorithms and the data will be made publicly available through online repositories embedded in European research infrastructures. The DeeperSense consortium consists of renowned experts in robotics and marine robotics, artificial

Field of science

• /engineering and technology/electrical engineering, electronic engineering, information engineering/electronic engineering/robotics/autonomous robots
• /natural sciences/computer and information sciences/artificial intelligence
• /engineering and technology/electrical engineering, electronic engineering, information engineering/electronic engineering/sensors

Programme(s)

• H2020-EU.2.1.1. - INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Topic(s)

• ICT-47-2020 - Research and Innovation boosting promising robotics applications

Call for proposal

H2020-ICT-2020-2
See other projects for this call

Funding Scheme

RIA - Research and Innovation action

Coordinator

DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH

Address

Trippstadter Strasse 122
67663 Kaiserslautern
Germany

Activity type

Research Organisations

EU contribution

€ 862 500

Participants (6)

UNIVERSITAT DE GIRONA

Spain

EU contribution

€ 475 000

Address

Placa Sant Domenec 3
17004 Girona

Activity type

Higher or Secondary Education Establishments

UNIVERSITY OF HAIFA

Israel

EU contribution

€ 619 462,50

Address

Abba Khushy Blvd Mount Carmel
31905 Haifa

Activity type

Higher or Secondary Education Establishments

KRAKEN ROBOTIK GMBH

Germany

EU contribution

€ 474 375

Address

Am Hohentorshafen 3
28197 Bremen

Activity type

Private for-profit entities (excluding Higher or Secondary Education Establishments)

BUNDESMINISTERIUM DES INNERN

Germany

EU contribution

€ 338 776,25

Address

Alt-moabit 101
10559 Berlin

Activity type

Public bodies (excluding Research Organisations and Secondary or Higher Education Establishments)

ISRAEL NATURE AND NATIONAL PARKS PROTECTION AUTHORITY

Israel

EU contribution

€ 102 500

Address

Am Ve Olamo Street 3
95463 Jerusalem

Activity type

Public bodies (excluding Research Organisations and Secondary or Higher Education Establishments)

TECNO AMBIENTE SL

Spain

EU contribution

€ 116 625

Address

Calle Industria 550-552
08918 Badalona Barcelona

Activity type

Private for-profit entities (excluding Higher or Secondary Education Establishments)