Learning Pixel-Perfect 3D Vision and Generative Modeling

Projektinformationen

PIPE

ID Finanzhilfevereinbarung: 866435

DOI

10.3030/866435

Projekt abgeschlossen

EK-Unterschriftsdatum 7 Februar 2020

Startdatum 1 September 2020

Enddatum 31 August 2025

Finanziert unter

• EXCELLENT SCIENCE - European Research Council (ERC)

Gesamtkosten

€ 1 858 013,00

EU-Beitrag

€ 1 858 013,00

Koordiniert durch

AALTO KORKEAKOULUSAATIO SR
Finland

Periodic Reporting for period 4 - PIPE (Learning Pixel-Perfect 3D Vision and Generative Modeling)

Berichtszeitraum: 2025-03-01 bis 2025-08-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

PIPE seeks to advance computers' capability to reason about the physical world by looking at nothing but pictures. Moreover, PIPE seeks to enable this reasoning to stand on solid, physically meaningful quantities. Concretely, PIPE studies both the problem of understanding what shapes, materials, lights, objects, etc., appear in pictures of real-world scenes, as well as the problem of generative modeling, i.e. learning "a recipe" for the world.

Solutions to these fundamental problems in computer vision, computer graphics, and machine learning are relevant for a large number of applications from computer graphics, augmented and virtual reality to robotics; clearly, truly understanding the messy complexity of the real world must be necessary, e.g. for reliable, safe autonomous machines to emerge.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

PIPE seeks to advance the state of the art on three axes: (1) understanding the shapes in the world; (2) understanding the visual complexity of the surface materials in the world; and (3) learning generative computational recipes for things in the world. The most significant achievement since the beginning of the project is a result in the final direction that combines intuitive, controllable computational components with modern machine learning techniques. This enables photorealistic generative modeling in terms of physically meaningful concepts and quantities while requiring no human supervision.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

PIPE is expected to provide breakthrough results in how well computer vision and generative algorithms can understand our three-dimensional world.

Overview of the approach in "A hybrid generator architecture for controllable face synthesis"