Analysis and Representation of Complex Activities in Videos

Project Information

ARCA

Grant agreement ID: 677650

DOI

10.3030/677650

Project closed

EC signature date 1 March 2016

Start date 1 June 2016

End date 31 May 2021

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 499 875,00

EU contribution

€ 1 499 875,00

Coordinated by

RHEINISCHE FRIEDRICH-WILHELMS-UNIVERSITAT BONN
Germany

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Book chapters (10)

Joint Viewpoint and Keypoint Estimation with Real and Synthetic Data

Author(s): Pau Panareda Busto, Juergen Gall
Published in: Pattern Recognition - 41st DAGM German Conference, DAGM GCPR 2019, Dortmund, Germany, September 10–13, 2019, Proceedings, Issue 11824, 2019, Page(s) 107-121, ISBN 978-3-030-33675-2
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-33676-9_8

Adversarial Synthesis of Human Pose from Text

Author(s): Yifei Zhang, Rania Briq, Julian Tanke, Juergen Gall
Published in: Pattern Recognition - 42nd DAGM German Conference, DAGM GCPR 2020, Tübingen, Germany, September 28 – October 1, 2020, Proceedings, Issue 12544, 2021, Page(s) 145-158, ISBN 978-3-030-71277-8
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-71278-5_11

Recursive Bayesian Filtering for Multiple Human Pose Tracking from Multiple Cameras

Author(s): Oh-Hun Kwon, Julian Tanke, Juergen Gall
Published in: Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30 – December 4, 2020, Revised Selected Papers, Part II, Issue 12623, 2021, Page(s) 438-453, ISBN 978-3-030-69531-6
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-69532-3_27

Iterative Greedy Matching for 3D Human Pose Tracking from Multiple Views

Author(s): Julian Tanke, Juergen Gall
Published in: Pattern Recognition - 41st DAGM German Conference, DAGM GCPR 2019, Dortmund, Germany, September 10–13, 2019, Proceedings, Issue 11824, 2019, Page(s) 537-550, ISBN 978-3-030-33675-2
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-33676-9_38

Discovering Multi-label Actor-Action Association in a Weakly Supervised Setting

Author(s): Sovan Biswas, Juergen Gall
Published in: Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30 – December 4, 2020, Revised Selected Papers, Part V, Issue 12626, 2021, Page(s) 547-561, ISBN 978-3-030-69540-8
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-69541-5_33

Spatio-temporal Channel Correlation Networks for Action Classification

Author(s): Ali Diba, Mohsen Fayyaz, Vivek Sharma, M. Mahdi Arzani, Rahman Yousefzadeh, Juergen Gall, Luc Van Gool
Published in: Computer Vision – ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IV, Issue 11208, 2018, Page(s) 299-315, ISBN 978-3-030-01224-3
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-01225-0_18

Hand Pose Estimation via Latent 2.5D Heatmap Regression

Author(s): Umar Iqbal, Pavlo Molchanov, Thomas Breuel, Juergen Gall, Jan Kautz
Published in: Computer Vision – ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XI, Issue 11215, 2018, Page(s) 125-143, ISBN 978-3-030-01251-9
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-01252-6_8

Multi-person Pose Estimation with Local Joint-to-Person Associations

Author(s): Umar Iqbal, Juergen Gall
Published in: Computer Vision – ECCV 2016 Workshops, 2016, Page(s) 627-642, ISBN 978-3-319-48881-3
Publisher: Springer International Publishing
DOI: 10.1007/978-3-319-48881-3_44

AVID: Adversarial Visual Irregularity Detection

Author(s): Mohammad Sabokrou, Masoud Pourreza, Mohsen Fayyaz, Rahim Entezari, Mahmood Fathy, Jürgen Gall, Ehsan Adeli
Published in: Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part VI, Issue 11366, 2019, Page(s) 488-505, ISBN 978-3-030-20875-2
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-20876-9_31

Long-Term Anticipation of Activities with Cycle Consistency

Author(s): Yazan Abu Farha, Qiuhong Ke, Bernt Schiele, Juergen Gall
Published in: Pattern Recognition - 42nd DAGM German Conference, DAGM GCPR 2020, Tübingen, Germany, September 28 – October 1, 2020, Proceedings, Issue 12544, 2021, Page(s) 159-173, ISBN 978-3-030-71277-8
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-71278-5_12

Conference proceedings (33)

MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

Author(s): Yazan Abu Farha, Jurgen Gall
Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Page(s) 3570-3579, ISBN 978-1-7281-3293-8
Publisher: IEEE
DOI: 10.1109/cvpr.2019.00369

Level Selector Network for Optimizing Accuracy-Specificity Trade-Offs

Author(s): Ahsan Iqbal, Juergen Gall
Published in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, Page(s) 1466-1473, ISBN 978-1-7281-5023-9
Publisher: IEEE
DOI: 10.1109/iccvw.2019.00184

Temporal Action Segmentation From Timestamp Supervision

Author(s): Zhe Li, Yazan Abu Farha, Jurgen Gall
Published in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, Page(s) 8365-8374
Publisher: IEEE

Uncertainty-Aware Anticipation of Activities

Author(s): Yazan Abu Farha, Juergen Gall
Published in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, Page(s) 1197-1204, ISBN 978-1-7281-5023-9
Publisher: IEEE
DOI: 10.1109/iccvw.2019.00151

Human Motion Prediction via Spatio-Temporal Inpainting

Author(s): Alejandro Hernandez, Jurgen Gall, Francesc Moreno
Published in: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, Page(s) 7133-7142, ISBN 978-1-7281-4803-8
Publisher: IEEE
DOI: 10.1109/iccv.2019.00723

Self-supervised Keypoint Correspondences for Multi-person Pose Estimation and Tracking in Videos

Author(s): Umer Rafi, Andreas Doering, Bastian Leibe, Juergen Gall
Published in: Computer Vision – ECCV 2020 - 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Issue 12365, 2020, Page(s) 36-52, ISBN 978-3-030-58565-5
Publisher: Springer
DOI: 10.1007/978-3-030-58565-5_3

Recurrent Residual Learning for Action Recognition

Author(s): Ahsan Iqbal, Alexander Richard, Hilde Kuehne, Juergen Gall
Published in: German Conference on Pattern Recognition, 2017, Page(s) 126-137
Publisher: Springer International Publishing
DOI: 10.1007/978-3-319-66709-6_11

Weakly Supervised Action Learning with RNN Based Fine-to-Coarse Modeling

Author(s): Alexander Richard, Hilde Kuehne, Juergen Gall
Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Page(s) 1273-1282, ISBN 978-1-5386-0457-1
Publisher: IEEE
DOI: 10.1109/cvpr.2017.140

PoseTrack: Joint Multi-person Pose Estimation and Tracking

Author(s): Umar Iqbal, Anton Milan, Juergen Gall
Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Page(s) 4654-4663, ISBN 978-1-5386-0457-1
Publisher: IEEE
DOI: 10.1109/cvpr.2017.495

Pose for Action - Action for Pose

Author(s): Umar Iqbal, Martin Garbade, Juergen Gall
Published in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, Page(s) 438-445, ISBN 978-1-5090-4023-0
Publisher: IEEE
DOI: 10.1109/fg.2017.61

Open Set Domain Adaptation

Author(s): Pau Panareda Busto, Juergen Gall
Published in: 2017 IEEE International Conference on Computer Vision (ICCV), 2017, Page(s) 754-763, ISBN 978-1-5386-1032-9
Publisher: IEEE
DOI: 10.1109/iccv.2017.88

PoseTrack: A Benchmark for Human Pose Estimation and Tracking

Author(s): Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, Bernt Schiele
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 5167-5176, ISBN 978-1-5386-6420-9
Publisher: IEEE
DOI: 10.1109/cvpr.2018.00542

Action Sets: Weakly Supervised Action Segmentation Without Ordering Constraints

Author(s): Alexander Richard, Hilde Kuehne, Juergen Gall
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 5987-5996, ISBN 978-1-5386-6420-9
Publisher: IEEE
DOI: 10.1109/cvpr.2018.00627

When will you do what? - Anticipating Temporal Occurrences of Activities

Author(s): Yazan Abu Farha, Alexander Richard, Juergen Gall
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 5343-5352, ISBN 978-1-5386-6420-9
Publisher: IEEE
DOI: 10.1109/cvpr.2018.00560

NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

Author(s): Alexander Richard, Hilde Kuehne, Ahsan Iqbal, Juergen Gall
Published in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Page(s) 7386-7395, ISBN 978-1-5386-6420-9
Publisher: IEEE
DOI: 10.1109/cvpr.2018.00771

Direct Shot Correspondence Matching

Author(s): Umer Rafi, Juergen Gall, Bastian Leibe
Published in: British Machine Vision Conference, 2018
Publisher: BMVA

JointFlow: Temporal Flow Fields for Multi Person Pose Estimation

Author(s): Andreas Doering, Umar Iqbal, Juergen Gall
Published in: British Machine Vision Conference, 2018
Publisher: BMVA

Convolutional Simplex Projection Network for Weakly Supervised Semantic Segmentation

Author(s): Rania Briq, Michael Moeller, Juergen Gall
Published in: British Machine Vision Conference, 2018
Publisher: BMVA

Structural Recurrent Neural Network (SRNN) for Group Activity Analysis

Author(s): Sovan Biswas, Juergen Gall
Published in: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, Page(s) 1625-1632, ISBN 978-1-5386-4886-5
Publisher: IEEE
DOI: 10.1109/wacv.2018.00180

An Efficient Convolutional Network for Human Pose Estimation

Author(s): Umer Rafi, Ilya Kostrikov, Juergen Gall, Bastian Leibe
Published in: British Machine Vision Conference, 2016
Publisher: BMVA Press

Handcrafting vs Deep Learning: An Evaluation of NTraj+ Features for Pose Based Action Recognition

Author(s): Martin Garbade, Juergen Gall
Published in: Workshop New Challenges in Neural Computation, 2016
Publisher: Thomas Villmann, Frank-Michael Schleif

SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation

Author(s): Mohsen Fayyaz, Jurgen Gall
Published in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Page(s) 498-507, ISBN 978-1-7281-7168-5
Publisher: IEEE
DOI: 10.1109/cvpr42600.2020.00058

Hierarchical Graph-Rnns for Action Detection of Multiple Activities

Author(s): Sovan Biswas, Yaser Souri, Juergen Gall
Published in: 2019 IEEE International Conference on Image Processing (ICIP), 2019, Page(s) 1-5, ISBN 978-1-5386-6249-6
Publisher: IEEE
DOI: 10.1109/icip.2019.8803650

You Only Need Adversarial Supervision for Semantic Image Synthesis

Author(s): Edgar Schönfeld, Vadim Sushko, Dan Zhang, Juergen Gall, Bernt Schiele, Anna Khoreva
Published in: International Conference on Learning Representations, 2021
Publisher: OpenReview.net

Unsupervised Video Representation Learning by Bidirectional Feature Prediction

Author(s): Nadine Behrmann, Juergen Gall, Mehdi Noroozi
Published in: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), 2021, Page(s) 1669-1678, ISBN 978-1-6654-0477-8
Publisher: IEEE
DOI: 10.1109/wacv48630.2021.00171

What Object Should I Use? - Task Driven Object Detection

Author(s): Johann Sawatzky, Yaser Souri, Christian Grund, Jurgen Gall
Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Page(s) 7597-7606, ISBN 978-1-7281-3293-8
Publisher: IEEE
DOI: 10.1109/cvpr.2019.00779

Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videos

Author(s): Olga Zatsarynna, Yazan Abu Farha, Juergen Gall
Published in: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021
Publisher: IEEE

Enhancing Temporal Action Localization with Transfer Learning from Action Recognition

Author(s): Ahsan Iqbal, Alexander Richard, Juergen Gall
Published in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, Page(s) 1533-1540, ISBN 978-1-7281-5023-9
Publisher: IEEE
DOI: 10.1109/iccvw.2019.00191

Cross-Modal Knowledge Distillation for Action Recognition

Author(s): Fida Mohammad Thoker, Juergen Gall
Published in: 2019 IEEE International Conference on Image Processing (ICIP), 2019, Page(s) 6-10, ISBN 978-1-5386-6249-6
Publisher: IEEE
DOI: 10.1109/icip.2019.8802909

Unsupervised Learning of Action Classes With Continuous Temporal Embedding

Author(s): Anna Kukleva, Hilde Kuehne, Fadime Sener, Jurgen Gall
Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Page(s) 12058-12066, ISBN 978-1-7281-3293-8
Publisher: IEEE
DOI: 10.1109/cvpr.2019.01234

Large Scale Holistic Video Understanding

Author(s): Ali Diba, Mohsen Fayyaz, Vivek Sharma, Manohar Paluri, Jürgen Gall, Rainer Stiefelhagen, Luc Van Gool
Published in: Computer Vision – ECCV 2020 - 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V, Issue 12350, 2020, Page(s) 593-610, ISBN 978-3-030-58557-0
Publisher: Springer International Publishing
DOI: 10.1007/978-3-030-58558-7_35

3D CNNs With Adaptive Temporal Feature Resolutions

Author(s): Mohsen Fayyaz, Emad Bahrami, Ali Diba, Mehdi Noroozi, Ehsan Adeli, Luc Van Gool, Jurgen Gall
Published in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, Page(s) 4731-4740
Publisher: IEEE

Harvesting Information from Captions for Weakly Supervised Semantic Segmentation

Author(s): Johann Sawatzky, Debayan Banerjee, Juergen Gall
Published in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019, Page(s) 4481-4490, ISBN 978-1-7281-5023-9
Publisher: IEEE
DOI: 10.1109/iccvw.2019.00549

Peer reviewed articles (8)

MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation

Author(s): Shi-Jie Li, Yazan AbuFarha, Yun Liu, Ming-Ming Cheng, Juergen Gall
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, Page(s) 1-1, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2020.3021756

Open Set Domain Adaptation for Image and Action Recognition

Author(s): Pau Panareda Busto, Ahsan Iqbal, Juergen Gall
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 42/2, 2020, Page(s) 413-429, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2018.2880750

A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation

Author(s): Hilde Kuehne, Alexander Richard, Juergen Gall
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 42/4, 2020, Page(s) 765-779, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2018.2884469

Weakly supervised learning of actions from transcripts

Author(s): Hilde Kuehne, Alexander Richard, Juergen Gall
Published in: Computer Vision and Image Understanding, Issue 163, 2017, Page(s) 78-89, ISSN 1077-3142
Publisher: Academic Press
DOI: 10.1016/j.cviu.2017.06.004

A bag-of-words equivalent recurrent neural network for action recognition

Author(s): Alexander Richard, Juergen Gall
Published in: Computer Vision and Image Understanding, Issue 156, 2017, Page(s) 79-91, ISSN 1077-3142
Publisher: Academic Press
DOI: 10.1016/j.cviu.2016.10.014

A dual-source approach for 3D human pose estimation from single images

Author(s): Umar Iqbal, Andreas Doering, Hashim Yasin, Björn Krüger, Andreas Weber, Juergen Gall
Published in: Computer Vision and Image Understanding, Issue 172, 2018, Page(s) 37-49, ISSN 1077-3142
Publisher: Academic Press
DOI: 10.1016/j.cviu.2018.03.007

Fast Weakly Supervised Action Segmentation Using Mutual Consistency

Author(s): Yaser Souri, Mohsen Fayyaz, Luca Minciullo, Gianpiero Francesca, Juergen Gall
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, Page(s) 1-1, ISSN 0162-8828
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpami.2021.3089127

Pose Refinement Graph Convolutional Network for Skeleton-Based Action Recognition

Author(s): Shijie Li, Jinhui Yi, Yazan Abu Farha, Juergen Gall
Published in: IEEE Robotics and Automation Letters, Issue 6/2, 2021, Page(s) 1028-1035, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2021.3056361