Objective Over the past 5 years, deep learning has exercised a tremendous and transformational effect on the field of computer vision. However, deep neural networks (DNNs) can only realize their full potential when applied in an end-to-end manner, i.e. when every stage of the processing pipeline is differentiable with respect to the network’s parameters, such that all of those parameters can be optimized together. Such end-to-end learning solutions are still rare for computer vision problems, in particular for dynamic visual scene understanding tasks. Moreover, feed-forward processing, as done in most DNN-based vision approaches, is only a tiny fraction of what the human brain can do. Feedback processes, temporal information processing, and memory mechanisms form an important part of our human scene understanding capabilities. Those mechanisms are currently underexplored in computer vision.The goal of this proposal is to remove this bottleneck and to design end-to-end deep learning approaches that can realize the full potential of DNNs for dynamic visual scene understanding. We will make use of the positive interactions and feedback processes between multiple vision modalities and combine them to work towards a common goal. In addition, we will impart deep learning approaches with a notion of what it means to move through a 3D world by incorporating temporal continuity constraints, as well as by developing novel deep associative and spatial memory mechanisms.The results of this research will enable deep neural networks to reach significantly improved dynamic scene understanding capabilities compared to today’s methods. This will have an immediate positive effect for applications in need for such capabilities, most notably for mobile robotics and intelligent vehicles. Fields of science natural sciencescomputer and information sciencesartificial intelligencecomputer visionnatural sciencescomputer and information sciencesartificial intelligencemachine learningdeep learningengineering and technologyelectrical engineering, electronic engineering, information engineeringelectronic engineeringroboticsnatural sciencescomputer and information sciencesdata sciencedata processingnatural sciencescomputer and information sciencesartificial intelligencecomputational intelligence Programme(s) H2020-EU.1.1. - EXCELLENT SCIENCE - European Research Council (ERC) Main Programme Topic(s) ERC-2017-COG - ERC Consolidator Grant Call for proposal ERC-2017-COG See other projects for this call Funding Scheme ERC-COG - Consolidator Grant Host institution RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN Net EU contribution € 2 000 000,00 Address TEMPLERGRABEN 55 52062 Aachen Germany See on map Region Nordrhein-Westfalen Köln Städteregion Aachen Activity type Higher or Secondary Education Establishments Links Contact the organisation Opens in new window Website Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Total cost € 2 000 000,00 Beneficiaries (1) Sort alphabetically Sort by Net EU contribution Expand all Collapse all RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN Germany Net EU contribution € 2 000 000,00 Address TEMPLERGRABEN 55 52062 Aachen See on map Region Nordrhein-Westfalen Köln Städteregion Aachen Activity type Higher or Secondary Education Establishments Links Contact the organisation Opens in new window Website Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Total cost € 2 000 000,00
