Infusing Data Fusion in Space Robotics

Project Information

InFuse

Grant agreement ID: 730014

DOI

10.3030/730014

Project closed

EC signature date 21 October 2016

Start date 1 November 2016

End date 31 January 2019

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies – Space

Total cost

€ 3 494 452,50

EU contribution

€ 3 493 952,50

Coordinated by

SPACE APPLICATIONS SERVICES NV
Belgium

Periodic Reporting for period 1 - InFuse (Infusing Data Fusion in Space Robotics)

Reporting period: 2016-11-01 to 2019-01-31

Summary of the context and overall objectives of the project

InFuse top level objective was to develop a data fusion framework offering a comprehensive set of data fusion techniques for perception and navigation, applicable to space robotics both in natural and man-made environments.
This overarching objective was fully achieved, with the open source release of the InFuse CDFF addressing both orbital and planetary applications, and both natural and artificial environment perception data handling. The set of data fusion capabilities implemented in the CDFF is quite comprehensive, allowing to address a number of relevant OOS and planetary robotics scenarios.
The first Specific Objective (SO1) was aiming to develop essential support functions for data fusion, and have them robust enough to fit in wide range of perception and navigation algorithms. A number of such fundamental data fusion algorithms was implemented (e.g. variations of Kalman Filter), and used in most perception and navigation algorithms released in the CDFF. Their robustness and performances were assessed only at DFPC level, not at individual (elementary) DFN algorithm level.
SO2 deals with perception related data fusion techniques. All the identified data fusion capabilities were integrated in the framework: target tracking and models reconstruction in particular. Alternative sensors data were also investigated, but more experimentally – related software was not integrated in the CDFF baseline release of InFuse.
SO3 addresses data fusion techniques for navigation. Localization in both structured and unstructured environments were tackled and covered in InFuse. For what concerns orbital applications, focus was on close to mid-range navigation.
In SO4, we proposed to develop a suite of tools supporting the access to data fusion techniques and products in the framework. 2 complementary data products management tools were implemented to handle data products in the frameworks.
SO5 dealt with the implementation of 2 reference implementations (RI) of InFuse: one orbital, and one planetary. The orbital RI was validated mainly with DLR OOS facilities and related data sets, while the planetary RI was validated with datasets obtained with the PEL facility of DLR, as well as with DFKI’s Sherpa platform (extended with a modular perception bench developed by DLR) and the Mana and Minnie rover platform of CNRS.
The dissemination of InFuse was good (6 peer reviewed papers). The consortium is now in the process of leveraging the results to publish journal papers (2 undergoing) and peer reviewed papers.
The InFuse CDFF was eventually released as an Open Source software on Gitlab, opening it to a wide community of roboticist – the framework is Space Robotics oriented and flavored, but definitely extends to a much larger community needs. We expect both space robotics and non-space robotics community to uptake InFuse’s results in various robotic applications. Gitlab link to the open source release: https://github.com/H2020-InFuse

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The following activity timeline describes the progress of the project:
M1 - M3: SRR
The first phase consisted of identifying scenarios for the orbital and planetary scenarios to be covered within the context of InFuse. Partners identified preliminary data fusion techniques to address these scenarios along with required sensors and desired performance metrics. From this information, the software requirements document were drafted with inputs from all partners.
M4 - M9: PDR
Preliminary design of CDFF software components were extensively discussed to arrive at an advanced architecture of the framework. There was the need to carry out early trade-off analysis with potential candidate solutions for the required data fusion functionalities. The consortium carefully check whether existing software solutions can be reused, reproduced or adapted, as much as possible. The resulting advanced architecture specification and ICD would serve as the reference to start prototyping data fusion and framework components.
M10 - M15: CDR
The final design WP focused on detailed design of the orbital and planetary reference implementation and its related EGSEs for supporting internal testing, verification & validation of CDFF-Core DFNs and DFPCs. The DLR OOS-Sim setup was analysed for potential adaptions with respect to sensors, calibration, data formats and scenario operations. Simultaneously, the PEL mars analogue with ExoMars BB2 and HCRU sensor package was analysed in a similar approach for planetary scenarios.
M16 - M22: TRR
This period was the core implementation phase of CDFF-Core, Dev and Support software packages. DFNs and DFPCs related to mapping, relative and absolute localisation, 3D detection and tracking, 3D reconstruction and model based approaches were developed as per the common interfaces and guidelines established in the project. A continuous integration framework ensured that contributions were of the desired quality before finalisation. CDFF Support software consisted of map and pose fused data product management and orchestrator. CDFF-Dev tools for offline log replay, visualisation, middleware facilitation was developed and tested with sample data sets.
M23 - M27 - FA
The last phase of the project consisted of verification & validation of CDFF software with OG6 facilities. For the orbital track, OOS-Sim was used as the reference EGSE for demonstrating the capabilities Orbital RI while tracking the accuracy and performance of close and mid-range scenarios for on orbit servicing. For the planetary track, the first V&V was carried out using data sets from ExoMars BB2 with HCRU sensor package. Following this, the Sherpa-HCRU and Mana & Minnie robots were used to collected outdoor analogue data sets during the Morocco campaign. DFPCs were evaluated against these data sets to capture the detailed performance, accuracy and run time characteristics compared against the ground truth.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

In space robotics, a wide range of data fusion techniques are required to accomplish challenging objectives for exploration, science and commercial purposes. This includes navigation for planetary and orbital robotics, scientific data gathering, and on-orbit spacecraft servicing applications. InFuse aims to develop a comprehensive open-source data fusion toolset to combine and interpret sensory data from multiple robotic sensors, referred as a Common Data Fusion Framework (CDFF). This project represents our effort to move robotic software development practices to the next step: a standardized and comprehensive development environment for industrial applications, with particular focus on space applications which components can be connected, tested offline, evaluated and deployed in any preferred robotic framework, including those devised for space applications.
The Common Data Fusion Framework (CDFF) contains the following features: (i)a library of reusable Data Fusion Node (DFN) modules implementing sensor fusion algorithms and validated for industrial applications, (ii) easy methods to quickly and reliably combine the DFNs into software modules(Data Fusion Processing Compounds, or DFPCs) meant for sensor data processing, (iii) tools for data collection and testing of the DFPCs, (iv) suitable conversion mechanisms between different representations of the sensor data, and (v)convenient integration with robotic control frameworks such as ESROCOS, ROS, ROCK, GenoM etc.

OOS-Sim at DLR

Aerial Digital Elevation Map of the test site in Morocco

Sherpa with HCRU

Registration of Lidar Point Clouds

Digital Elevation Model of the PEL indoor analog at DLR

Detection and tracking of a target satellite using OOS-Sim at DLR

Detection and tracking of a target satellite using OOS-Sim at DLR

ExoMars BB2 rover used to collect data for InFuse

Mana and Minnie in Morocco