3D Reloaded: Novel Algorithms for 3D Shape Inference and Analysis

Projektinformationen

3D Reloaded

ID Finanzhilfevereinbarung: 649323

DOI

10.3030/649323

Projekt abgeschlossen

EK-Unterschriftsdatum 5 Mai 2015

Startdatum 1 September 2015

Enddatum 31 August 2020

Finanziert unter

• EXCELLENT SCIENCE - European Research Council (ERC)

Gesamtkosten

€ 2 000 000,00

EU-Beitrag

€ 2 000 000,00

Koordiniert durch

TECHNISCHE UNIVERSITAET MUENCHEN
Germany

Periodic Reporting for period 4 - 3D Reloaded (3D Reloaded: Novel Algorithms for 3D Shape Inference and Analysis)

Berichtszeitraum: 2020-03-01 bis 2020-08-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

The isssue adddressed in the project "3D Reloaded" is the development of novel algorithms for 3D reconstruction from cameras and for 3D shape analysis. Both of these problems are of central importance for machines to understand the 3D world observed with a camera.

This is important for society as it affects a number of application domains: For the development of self-driving cars, the autonomous cars needs to understand what happens around (and in particular) in front of the car. For augmented reality, one is interested in recovering the 3D structure of our world fIn the development of medical image analysis, algorithms need to interpret and analyze changes of inner organs represented in form of 3D shapes.

The overall objectives of the project are:
1. the development of algorithms for reconstructing 3D structures from monocular and from RGB-D cameras with a particular focus on dense and realtime capable methods,
2. the development of algorithms for 3D shape analysis, including problems such as measuring shape similarity, matching of shapes and the semantic analysis of shapes.
3. For the next phase of the project, a central objective is the development of 3D shape priors for 3D reconstruction.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

We developed a number of algorithms for:
- reconstructing 3D scenes from monocular cameras with known pose,
- reconstrucing 3D scenes from RGB-D cameras with known pose,
- jointly reconstructing 3D scenes and camera trajectory from moving monocular cameras (visual odometry / simultaneous localization and mapping),
- jointly reconstructing 3D scenes and camera trajectory from moving RGB-D cameras,
- geometric shape priors for 3D reonstruction.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

The algorithms we developed were in many ways beyond the state of the art - in terms of accuracy, in terms of robustness, in terms of scalability, in terms of runtime, etc.

The main emphasis and expected results until the end of the project is the development of shape priors for 3D reconstruction. This is one of the topics we are currently working on.

Fight ill-posedness with ill-posedness: Single-shot variational, Haefner et al., CVPR 2018.

Divergence-free Shape Interpolation and Correspondence, Eisenberger, Laehner, Cremers, CVPR 2019.

Direct Sparse Odometry, Engel, Koltun, Cremers, PAMI 2018.