CORDIS
EU research results

CORDIS

English EN
Deep Learning for Dynamic 3D Visual Scene Understanding

Deep Learning for Dynamic 3D Visual Scene Understanding

Objective

Over the past 5 years, deep learning has exercised a tremendous and transformational effect on the field of computer vision. However, deep neural networks (DNNs) can only realize their full potential when applied in an end-to-end manner, i.e., when every stage of the processing pipeline is differentiable with respect to the network’s parameters, such that all of those parameters can be optimized together. Such end-to-end learning solutions are still rare for computer vision problems, in particular for dynamic visual scene understanding tasks. Moreover, feed-forward processing, as done in most DNN-based vision approaches, is only a tiny fraction of what the human brain can do. Feedback processes, temporal information processing, and memory mechanisms form an important part of our human scene understanding capabilities. Those mechanisms are currently underexplored in computer vision.

The goal of this proposal is to remove this bottleneck and to design end-to-end deep learning approaches that can realize the full potential of DNNs for dynamic visual scene understanding. We will make use of the positive interactions and feedback processes between multiple vision modalities and combine them to work towards a common goal. In addition, we will impart deep learning approaches with a notion of what it means to move through a 3D world by incorporating temporal continuity constraints, as well as by developing novel deep associative and spatial memory mechanisms.

The results of this research will enable deep neural networks to reach significantly improved dynamic scene understanding capabilities compared to today’s methods. This will have an immediate positive effect for applications in need for such capabilities, most notably for mobile robotics and intelligent vehicles.
Leaflet | Map data © OpenStreetMap contributors, Credit: EC-GISCO, © EuroGeographics for the administrative boundaries

Host institution

RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN

Address

Templergraben 55
52062 Aachen

Germany

Activity type

Higher or Secondary Education Establishments

EU Contribution

€ 2 000 000

Beneficiaries (1)

Sort alphabetically

Sort by EU Contribution

Expand all

RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN

Germany

EU Contribution

€ 2 000 000

Project information

Grant agreement ID: 773161

Status

Ongoing project

  • Start date

    1 April 2018

  • End date

    31 March 2023

Funded under:

H2020-EU.1.1.

  • Overall budget:

    € 2 000 000

  • EU contribution

    € 2 000 000

Hosted by:

RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN

Germany