Learning Generative 3D Scene Models for Training and Validating Intelligent Systems

Project Information

LEGO-3D

Grant agreement ID: 850533

DOI 10.3030/850533

EC signature date 22 October 2019

Start date 1 October 2020

End date 31 March 2026

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost € 1 467 500,00

EU contribution € 1 467 500,00

Coordinated by

EBERHARD KARLS UNIVERSITAET TUEBINGEN
Germany

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Conference proceedings (27)

ARAH: Animatable Volume Rendering of Articulated Human SDFs

Author(s): S. Wang, K. Schwarz, A. Geiger, S. Tang
Published in: European Conference on Computer Vision (ECCV), 2022
Publisher: Springer

KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs

Author(s): C. Reiser, S. Peng, Y. Liao, A. Geiger
Published in: International Conference on Computer Vision (ICCV), 2021
Publisher: IEEE/CVF

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

Author(s): Zijian Dong, Xu Chen, Jinlong Yang, Michael J. Black, Otmar Hilliges, Andreas Geiger
Published in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2024, Page(s) 14870-14881
Publisher: IEEE
DOI: 10.1109/iccv51070.2023.01370

SMD-Nets: Stereo Mixture Density Networks

Author(s): F. Tosi, Y. Liao, C. Schmitt, A. Geiger
Published in: Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Publisher: IEEE/CV

Hidden Biases of End-to-End Driving Models

Author(s): Bernhard Jaeger, Kashyap Chitta, Andreas Geiger
Published in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2024, Page(s) 8206-8215
Publisher: IEEE
DOI: 10.1109/iccv51070.2023.00757

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

Author(s): Oechsle, Michael; Peng, Songyou; Geiger, Andreas
Published in: International Conference on Computer Vision (ICCV), 2021
Publisher: IEEE/CVF
DOI: 10.1109/iccv48922.2021.00554

Shape As Points: A Differentiable Poisson Solver

Author(s): "Peng, Songyou; Jiang, Chiyu ""Max""; Liao, Yiyi; Niemeyer, Michael; Pollefeys, Marc; Geiger, Andreas"
Published in: Advances in Neural Information Processing Systems (NeurIPS), 2021
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2106.03452

IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing

Author(s): Shaofei Wang, Božidar Antić, Andreas Geiger, Siyu Tang
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 1877-1888
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.00184

MuRF: Multi-Baseline Radiance Fields

Author(s): Haofei XU, Anpei Chen, Yuedong Chen, Christos Sakaridis, Yulun Zhang, Marc Pollefeys, Andreas Geiger, Fisher Yu
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 20041-20050
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.01894

GOOD: Exploring geometric cues for detecting objects in an open world

Author(s): H. Huang, A. Geiger, D. Zhang
Published in: International Conference on Learning Representations (ICLR), 2023
Publisher: OpenReview

Mip-Splatting: Alias-Free 3D Gaussian Splatting

Author(s): Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 19447-19456
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.01839

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Author(s): Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, Shenghua Gao
Published in: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers '24, 2024, Page(s) 1-11
Publisher: ACM
DOI: 10.1145/3641519.3657428

GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields

Author(s): Geiger, Andreas; Niemeyer, Michael
Published in: Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Publisher: IEEE/CVF
DOI: 10.48550/arxiv.2011.12100

Multi-Modal Fusion Transformer for End-to-End Autonomous Driving

Author(s): Aditya Prakash, Kashyap Chitta, Andreas Geiger
Published in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, Page(s) 7073-7083
Publisher: IEEE
DOI: 10.1109/cvpr46437.2021.00700

NeLF-Pro: Neural Light Field Probes for Multi-Scale Novel View Synthesis

Author(s): Zinuo You, Andreas Geiger, Anpei Chen
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 19833-19843
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.01875

Projected GANs Converge Faster

Author(s): A. Sauer, K. Chitta, J. Müller, A. Geiger
Published in: Advances in Neural Information Processing Systems (NeurIPS), 2021
Publisher: Curran Associates, Inc.
DOI: 10.48550/arxiv.2111.01007

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets

Author(s): A. Sauer, K. Schwarz, A. Geiger
Published in: International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), 2022
Publisher: Association for Computing Machinery
DOI: 10.48550/arxiv.2202.00273

MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images

Author(s): S. Wang, M. Mihajlovic, Q. Ma, A. Geiger, S. Tang
Published in: Advances in Neural Information Processing Systems (NeurIPS), 2021
Publisher: Curran Associates, Inc.

GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers

Author(s): Takeru Miyato, Bernhard Jaeger, Max Welling, Andreas Geiger
Published in: International Conference on Learning Representations (ICLR), 2024
Publisher: OpenReview.net
DOI: 10.48550/arxiv.2310.10375

VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids

Author(s): K. Schwarz, A. Sauer, M. Niemeyer, Y. Liao, A. Geiger
Published in: Advances in Neural Information Processing Systems (NeurIPS), 2022
Publisher: Curran Associates, Inc.

Parting with Misconceptions about Learning-based Vehicle Motion Planning

Author(s): Dauner, Daniel; Hallgarten, Marcel; Geiger, Andreas; Chitta, Kashyap
Published in: Conference on Robot Learning, 2023
Publisher: Proceedings of Machine Learning Research
DOI: 10.48550/arxiv.2306.07962

Counterfactual Generative Networks

Author(s): Sauer, Axel; Geiger, Andreas
Published in: International Conference on Learning Representations (ICLR), 2021
Publisher: OpenReview
DOI: 10.48550/arxiv.2101.06046

3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting

Author(s): Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 5020-5030
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.00480

CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields

Author(s): Michael Niemeyer; Andreas Geiger
Published in: International Conference on 3D Vision (3DV), 2021
Publisher: IEEE Computer Society
DOI: 10.1109/3dv53792.2021.00103

HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting

Author(s): Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, Yiyi Liao
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 21336-21345
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.02016

GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs

Author(s): Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, Bernhard Schölkopf
Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, Page(s) 21295-21304
Publisher: IEEE
DOI: 10.1109/cvpr52733.2024.02012

MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction

Author(s): Z. Yu, S. Peng, M. Niemeyer, T. Sattler, A. Geiger
Published in: Advances in Neural Information Processing Systems (NeurIPS), 2022
Publisher: Curran Associates, Inc.

Peer reviewed articles (8)

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Author(s): Liao, Yiyi; Xie, Jun; Geiger, Andreas
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 1, 2022, ISSN 1939-3539
Publisher: IEEE Computer Society
DOI: 10.1109/tpami.2022.3179507

End-to-End Autonomous Driving: Challenges and Frontiers

Author(s): Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, Hongyang Li
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 46, 2024, Page(s) 10164-10183, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2024.3435937

Unifying Flow, Stereo and Depth Estimation

Author(s): Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, Andreas Geiger
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 45, 2023, Page(s) 13941-13958, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2023.3298645

Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis

Author(s): Christian Reiser, Stephan Garbin, Pratul Srinivasan, Dor Verbin, Richard Szeliski, Ben Mildenhall, Jonathan Barron, Peter Hedman, Andreas Geiger
Published in: ACM Transactions on Graphics, Issue 43, 2024, Page(s) 1-14, ISSN 0730-0301
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3658130

MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes

Author(s): Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, Peter Hedman
Published in: ACM Transactions on Graphics, Issue 42, 2024, Page(s) 1-12, ISSN 0730-0301
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3592426

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

Author(s): K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, A. Geiger
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2022.3200245

Towards Scalable Multi-View Reconstruction of Geometry and Materials

Author(s): Carolin Schmitt, Božidar Antić, Andrei Neculai, Joo Ho Lee, Andreas Geiger
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 45, 2024, Page(s) 15850-15869, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2023.3314348

Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes

Author(s): Zehao Yu, Torsten Sattler, Andreas Geiger
Published in: ACM Transactions on Graphics, Issue 43, 2024, Page(s) 1-13, ISSN 0730-0301
Publisher: Association for Computing Machinary, Inc.
DOI: 10.1145/3687937

Book chapters (3)

LaRa: Efficient Large-Baseline Radiance Fields

Author(s): Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger
Published in: Lecture Notes in Computer Science, Computer Vision – ECCV 2024, 2024, Page(s) 338-355
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-73247-8_20

Efficient Depth-Guided Urban View Synthesis

Author(s): Sheng Miao, Jiaxin Huang, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Andreas Geiger, Yiyi Liao
Published in: Lecture Notes in Computer Science, Computer Vision – ECCV 2024, 2025, Page(s) 90-107
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-73404-5_6

SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic

Author(s): Kashyap Chitta, Daniel Dauner, Andreas Geiger
Published in: Lecture Notes in Computer Science, Computer Vision – ECCV 2024, 2024, Page(s) 57-74
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-73235-5_4