Deep Learning for Dynamic 3D Visual Scene Understanding

Project Information

DeeViSe

Grant agreement ID: 773161

DOI

10.3030/773161

Project closed

EC signature date 19 March 2018

Start date 1 April 2018

End date 31 March 2023

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 2 000 000,00

EU contribution

€ 2 000 000,00

Coordinated by

RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEN
Germany

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Peer reviewed articles (3)

Track to Reconstruct and Reconstruct to Track

Author(s): Jonathon Luiten, Tobias Fischer, Bastian Leibe
Published in: IEEE Robotics and Automation Letters, Issue 5/2, 2020, Page(s) 1803-1810, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2020.2969183

MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose Estimation

Author(s): Istvan Sarandi, Timm Linder, Kai Oliver Arras, Bastian Leibe
Published in: IEEE Transactions on Biometrics, Behavior, and Identity Science, Issue 3/1, 2021, Page(s) 16-30, ISSN 2637-6407
Publisher: IEEE
DOI: 10.1109/tbiom.2020.3037257

HOTA: A Higher Order Metric for Evaluating Multi-object Tracking

Author(s): Jonathon Luiten, Aljos̆a Os̆ep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixé, Bastian Leibe
Published in: International Journal of Computer Vision, Issue 129/2, 2021, Page(s) 548-578, ISSN 0920-5691
Publisher: Kluwer Academic Publishers
DOI: 10.1007/s11263-020-01375-2

Conference proceedings (39)

From Points to Multi-Object 3D Reconstruction

Author(s): Francis Engelmann, Konstantinos Rematas, Bastian Leibe, Vittorio Ferrari
Published in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, ISBN 978-1-6654-4509-2
Publisher: IEEE
DOI: 10.1109/cvpr46437.2021.00456

Mask3D: Mask Transformer for 3D Semantic Instance Segmentation

Author(s): Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, Bastian Leibe
Published in: 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023
Publisher: IEEE
DOI: 10.1109/icra48891.2023.10160590

UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking

Author(s): Jonathon Luiten, Idil Esen Zulfikar, Bastian Leibe
Published in: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, Page(s) 1989-1998, ISBN 978-1-7281-6553-0
Publisher: IEEE
DOI: 10.1109/wacv45572.2020.9093285

TarViS: A Unified Approach for Target-based Video Segmentation

Author(s): Ali Athar, Alexander Hermans, Jonathon Luiten, Deva Ramanan, Bastian Leibe
Published in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Publisher: IEEE
DOI: 10.48550/arxiv.2301.02657

3D Segmentation of Humans in Point Clouds with Synthetic Data

Author(s): Ayça Takmaz, Jonas Schult, Irem Kaftan, Mertcan Akçay, Bastian Leibe, Robert Sumner, Francis Engelmann, Siyu Tang
Published in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Publisher: IEEE
DOI: 10.48550/arxiv.2212.00786

Differentiable Soft-Masked Attention

Author(s): Ali Athar, Jonathon Luiten, Alexander Hermans, Deva Ramanan, Bastian Leibe
Published in: 2022 CVRP Transformers for Vision Workshop, 2022
Publisher: arxiv
DOI: 10.48550/arxiv.2206.00182

Siam R-CNN: Visual Tracking by Re-Detection

Author(s): Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 6577-6587, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.00661

DynaMITe: Dynamic Query Bootstrapping for Multi-object Interactive Segmentation Transformer

Author(s): Amit Kumar Rana, Sabarinath Mahadevan, Alexander Hermans, Bastian Leibe
Published in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Publisher: IEEE
DOI: 10.48550/arxiv.2304.06668

BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video

Author(s): Ali Athar, Jonathon Luiten, Paul Voigtlaender, Tarasha Khurana, Achal Dave, Bastian Leibe, Deva Ramanan
Published in: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023
Publisher: IEEE
DOI: 10.1109/wacv56688.2023.00172

Making a Case for 3D Convolutions for Object Segmentation in Videos

Author(s): Mahadevan, Sabarinath; Athar, Ali; Ošep, Aljoša; Hennen, Sebastian; Leal-Taixé, Laura; Leibe, Bastian
Published in: British Machine Vision Conference (BMVC), 2020
Publisher: BMVA

Reducing the Annotation Effort for Video Object Segmentation Datasets

Author(s): Voigtlaender, Paul; Luo, Lishu; Yuan, Chun; Jiang, Yong; Leibe, Bastian
Published in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, Page(s) 3060-3069
Publisher: IEEE

HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Images

Author(s): Ali Athar, Jonathon Luiten, Alexander Hermans, Deva Ramanan, Bastian Leibe
Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, ISBN 978-1-6654-6946-3
Publisher: IEEE
DOI: 10.1109/cvpr52688.2022.00303

Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds

Author(s): Francis Engelmann, Theodora Kontogianni, Bastian Leibe
Published in: 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, Page(s) 9463-9469, ISBN 978-1-7281-7395-5
Publisher: IEEE
DOI: 10.1109/icra40945.2020.9197503

4D-StOP: Panoptic Segmentation of 4D LiDAR Using Spatio-Temporal Object Proposal Generation and Aggregation

Author(s): Lars Kreuzberg, Idil Esen Zülfikar, Sabarinath Mahadevan, Francis Engelmann, Bastian Leibe
Published in: 2022 ECCV Workshops, 2022
Publisher: Springer
DOI: 10.1007/978-3-031-25056-9_34

Reposing Humans by Warping 3D Features

Author(s): Markus Knoche, Istvan Sarandi, Bastian Leibe
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020, Page(s) 4499-4504, ISBN 978-1-7281-9360-1
Publisher: IEEE
DOI: 10.1109/cvprw50498.2020.00530

4D Generic Video Object Proposals

Author(s): Aljosa Osep, Paul Voigtlaender, Mark Weber, Jonathon Luiten, Bastian Leibe
Published in: 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, Page(s) 10031-10037, ISBN 978-1-7281-7395-5
Publisher: IEEE
DOI: 10.1109/icra40945.2020.9196949

DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes

Author(s): Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 8609-8619, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.00864

Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats

Author(s): Istvan Sarandi, Alexander Hermans, Bastian Leibe
Published in: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023
Publisher: IEEE
DOI: 10.1109/wacv56688.2023.00297

MOTS: Multi-Object Tracking and Segmentation

Author(s): Paul Voigtlaender, Michael Krause, Aljosa Osep, Jonathon Luiten, Berin Balachandar Gnana Sekar, Andreas Geiger, Bastian Leibe
Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Page(s) 7934-7943, ISBN 978-1-7281-3293-8
Publisher: IEEE
DOI: 10.1109/cvpr.2019.00813

Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds

Author(s): Francis Engelmann, Theodora Kontogianni, Jonas Schult, Bastian Leibe
Published in: Computer Vision – ECCV 2018 Workshops - Munich, Germany, September 8-14, 2018, Proceedings, Part III, Issue 11131, 2019, Page(s) 395-409, ISBN 978-3-030-11014-7
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-11015-4_29

Large-Scale Object Mining for Object Discovery from Unlabeled Video

Author(s): Aljosa Osep, Paul Voigtlaender, Jonathon Luiten, Stefan Breuers, Bastian Leibe
Published in: 2019 International Conference on Robotics and Automation (ICRA), 2019, Page(s) 5502-5508, ISBN 978-1-5386-6027-0
Publisher: IEEE
DOI: 10.1109/icra.2019.8793683

PReMVOS: Proposal-Generation, Refinement and Merging for Video Object Segmentation

Author(s): Jonathon Luiten, Paul Voigtlaender, Bastian Leibe
Published in: Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part IV, Issue 11364, 2019, Page(s) 565-580, ISBN 978-3-030-20869-1
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-20870-7_35

Iteratively Trained Interactive Segmentation

Author(s): Mahadevan, Sabarinath; Voigtlaender, Paul; Leibe, Bastian
Published in: British Machine Vision Conference (BMVC), 2018
Publisher: BMVA

Visual Person Understanding Through Multi-task and Multi-dataset Learning

Author(s): Kilian Pfeiffer, Alexander Hermans, István Sárándi, Mark Weber, Bastian Leibe
Published in: Pattern Recognition - 41st DAGM German Conference, DAGM GCPR 2019, Dortmund, Germany, September 10–13, 2019, Proceedings, Issue 11824, 2019, Page(s) 551-566, ISBN 978-3-030-33675-2
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-33676-9_39

AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects

Author(s): Johannes GroB, Aljosa Osep, Bastian Leibe
Published in: 2019 International Conference on 3D Vision (3DV), 2019, Page(s) 623-632, ISBN 978-1-7281-3131-3
Publisher: IEEE
DOI: 10.1109/3dv.2019.00074

Towards Large-Scale Video Video Object Mining

Author(s): Aljosa Osep, Paul Voigtlaender, Jonathon Luiten, Stefan Breuers, Bastian Leibe
Published in: ECCV 2018 Workshop on Interactive and Adaptive Learning in an Open World, 2018
Publisher: arXiv

Direct Shot Correspondence Matching

Author(s): Umer Rafi, Jürgen Gall, Bastian Leibe
Published in: British Machine Vision Conference (BMVC), 2018
Publisher: BMVA

Synthetic Occlusion Augmentation with Volumetric Heatmaps for the 2018 ECCV PoseTrack Challenge on 3D Human Pose Estimation

Author(s): Istvan Sarandi, Timm Linder, Kai Arras, Bastian Leibe
Published in: ECCV 2018 Workshops, 2018
Publisher: arXiv

Video Instance Segmentation 2019: A Winning Approach for Combined Detection, Segmentation, Classification and Tracking

Author(s): Jonathon Luiten, Philipp Torr, Bastian Leibe
Published in: The IEEE International Conference on Computer Vision (ICCV) Workshops, 2019
Publisher: Computer Vision Foundation

Exploring the Combination of PReMVOS, BoLTVOS and UnOVOST for the 2019 YouTube-VOS Challenge

Author(s): Jonathon Luiten, Paul Voigtlaender, Bastian Leibe
Published in: ICCV Workshops, 2019
Publisher: Computer Vision Foundation

Single-Shot Panoptic Segmentation

Author(s): Mark Weber, Jonathon Luiten, Bastian Leibe
Published in: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, Page(s) 8476-8483, ISBN 978-1-7281-6212-6
Publisher: IEEE
DOI: 10.1109/iros45743.2020.9341546

Metric-Scale Truncation-Robust Heatmaps for 3D Human Pose Estimation

Author(s): Istvan Sarandi, Timm Linder, Kai O. Arras, Bastian Leibe
Published in: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), 2020, Page(s) 407-414, ISBN 978-1-7281-3079-8
Publisher: IEEE
DOI: 10.1109/fg47880.2020.00108

Opening up Open World Tracking

Author(s): Yang Liu, Idil Esen Zulfikar, Jonathon Luiten, Achal Dave, Deva Ramanan, Bastian Leibe, Aljosa Osep, Laura Leal-Taixe
Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, ISBN 978-1-6654-6946-3
Publisher: IEEE
DOI: 10.1109/cvpr52688.2022.01846

A Single-Stage, Bottom-up Approach for Occluded VIS using Spatio-temporal Embeddings

Author(s): Ali Athar, Sabarinath Mahadevan, Aljosa Osep, Laura Leal-Taixe, Bastian Leibe
Published in: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, ISBN 978-1-6654-0191-3
Publisher: IEEE
DOI: 10.1109/iccvw54120.2021.00431

Mix3D: Out-of-Context Data Augmentation for 3D Scenes

Author(s): Alexey Nekrasov, Jonas Schult, Or Litany, Bastian Leibe, Francis Engelmann
Published in: 2021 International Conference on 3D Vision (3DV), 2021, ISBN 978-1-6654-2688-6
Publisher: IEEE
DOI: 10.1109/3dv53792.2021.00022

3D-MPA: Multi-Proposal Aggregation for 3D Semantic Instance Segmentation

Author(s): Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias NieBner
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 9028-9037, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.00905

Global Hierarchical Attention for 3D Point Cloud Analysis

Author(s): Dan Jia, Alexander Hermans, Bastian Leibe
Published in: DAGM German Conference on Pattern Recognition, 2022, ISBN 978-3-031-16787-4
Publisher: Springer
DOI: 10.1007/978-3-031-16788-1_17

D2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos

Author(s): Christian Schmidt, Ali Athar, Sabarinath Mahadevan, Bastian Leibe
Published in: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, ISBN 978-1-6654-0915-5
Publisher: IEEE
DOI: 10.1109/wacv51458.2022.00199

Point2Vec for Self-Supervised Representation Learning on Point Clouds

Author(s): Karim Abou Zeid, Jonas Schult, Alexander Hermans, Bastian Leibe
Published in: DAGM GCPR 2023, 2023
Publisher: Springer
DOI: 10.48550/arxiv.2303.16570

Book chapters (2)

3D Bird’s-Eye-View Instance Segmentation

Author(s): Cathrin Elich, Francis Engelmann, Theodora Kontogianni, Bastian Leibe
Published in: Pattern Recognition - 41st DAGM German Conference, DAGM GCPR 2019, Dortmund, Germany, September 10–13, 2019, Proceedings, Issue 11824, 2019, Page(s) 48-61, ISBN 978-3-030-33675-2
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-33676-9_4

STEm-Seg: Spatio-Temporal Embeddings for Instance Segmentation in Videos

Author(s): Ali Athar, Sabarinath Mahadevan, Aljos̆a Os̆ep, Laura Leal-Taixé, Bastian Leibe
Published in: Computer Vision – ECCV 2020 - 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI, Issue 12356, 2020, Page(s) 158-177, ISBN 978-3-030-58620-1
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-58621-8_10