Elastic Coordination for Scalable Machine Learning

Project Information

ScaleML

Grant agreement ID: 805223

DOI

10.3030/805223

Project closed

EC signature date 17 October 2018

Start date 1 March 2019

End date 29 February 2024

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 494 121,00

EU contribution

€ 1 494 121,00

Coordinated by

INSTITUTE OF SCIENCE AND TECHNOLOGY AUSTRIA
Austria

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Conference proceedings (18)

On the sample complexity of adversarial multi-source PAC learning

Author(s): Nikola Konstantinov, Elias Frantar, Dan Alistarh, Christoph H. Lampert
Published in: Proceedings of the 37th International Conference on Machine Learning (ICML 2020), 2020
Publisher: PMLR

Taming unbalanced training workloads in deep learning with partial collective operations

Author(s): Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo, Dan Alistarh, Torsten Hoefler
Published in: Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2020, Page(s) 45-61, ISBN 9781450368186
Publisher: ACM
DOI: 10.1145/3332466.3374528

The splay-list: a distribution-adaptive concurrent skip-list

Author(s): Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
Published in: Proceedings of the 34th International Symposium on Distributed Computing (DISC 2020), 2020
Publisher: LIPIcs
DOI: 10.4230/lipics.disc.2020.3

Non-blocking interpolation search trees with doubly-logarithmic running time

Author(s): Trevor Brown, Aleksandar Prokopec, Dan Alistarh
Published in: Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2020, Page(s) 276-291, ISBN 9781450368186
Publisher: ACM
DOI: 10.1145/3332466.3374542

AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks

Author(s): Alexandra Peste, Eugenia Iofinova, Adrian Vladu, Dan Alistarh
Published in: NeurIPS Proceedings, 2021
Publisher: NeurIPS

Towards Tight Communication Lower Bounds for Distributed Optimisation

Author(s): Janne Korhonen, Dan Alistarh
Published in: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021
Publisher: NeurIPS

Efficiency guarantees for parallel incremental algorithms under relaxed schedulers

Author(s): Dan Alistarh, Nikita Koval, Giorgi Nadiradze
Published in: 31st ACM Symposium on Parallelism in Algorithms and Architectures, 2019, Page(s) 145-154, ISBN 9781450361842
Publisher: ACM
DOI: 10.1145/3323165.3323201

Byzantine-Resilient Non-Convex Stochastic Gradient Descent

Author(s): Zeyuan Allen-Zhu, Faeze Ebrahimianghazani, Jerry Li, Dan Alistarh
Published in: Proceedings of the International Conference on Learning Representations (ICLR), 2021
Publisher: Openreview

Powerset convolutional neural networks

Author(s): Chris Wendler, Dan Alistarh, Markus Püschel
Published in: Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), Issue 32, 2019, Page(s) 927-938
Publisher: NIPS: Conference on Neural Information Processing Systems

WoodFisher: Efficient Second-Order Approximation for Neural Network Compression

Author(s): Sidak Pal Singh, Dan Alistarh
Published in: Proceedings of the International Conference on Neural Information Processing Systems, 2020
Publisher: NeurIPS

Communication-Efficient Federated Learning With Data and Client Heterogeneity

Author(s): Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze, Dan Alistarh
Published in: Proceedings of AISTATS 2024, 2024
Publisher: PMLR

Asynchronous Decentralized SGD with Quantized and Local Updates

Author(s): Giorgi Nadiradze, Amirmojtaba Sabour, Peter Davies, Shigang Li, Dan Alistarh
Published in: NeurIPS Proceedings, 2021
Publisher: NeurIPS Proceedings

Lower Bounds for Shared-Memory Leader Election under Bounded Write Contention

Author(s): Dan Alistarh, Rati Gelashvili, Giorgi Nadiradze
Published in: Proceedings of the 35th International Symposium on Distributed Computing (DISC 2021), 2021, Page(s) 4:1--4:17
Publisher: "Schloss Dagstuhl -- Leibniz-Zentrum f{\""u}r Informatik"
DOI: 10.4230/lipics.disc.2021.4

Relaxed Scheduling for Scalable Belief Propagation

Author(s): Vitaly Aksenov, Janne Korhonen, Dan Alistarh
Published in: Proceedings of the 36th International Conference on Neural Information Processing Systems, 2020
Publisher: NeurIPS

L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning

Author(s): Mohammadreza Alimohammadi, Ilia Markov, Elias Frantar, Dan Alistarh
Published in: Proceedings of MLSys 2024, 2024
Publisher: MLSys

SparCML - high-performance sparse communication for machine learning

Author(s): Cedric Renggli, Saleh Ashkboos, Mehdi Aghagolzadeh, Dan Alistarh, Torsten Hoefler
Published in: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2019, Page(s) 1-15, ISBN 9781450362290
Publisher: ACM
DOI: 10.1145/3295500.3356222

New Bounds For Distributed Mean Estimation and Variance Reduction

Author(s): Peter Davies, Vijaykrishna Gurunathan, Niusha Moshrefi, Saleh Ashkboos, Dan Alistarh
Published in: Proceedings of the International Conference on Learning Representations, 2021
Publisher: Openreview

Dynamic averaging load balancing on cycles

Author(s): Dan Alistarh, Giorgi Nadiradze, Amirmojtaba Sabour
Published in: Leibniz International Proceedings in Informatics, Issue 168/7, 2020, ISBN 9783959771382
Publisher: Schloss Dagstuhl - Leibniz-Zentrum für Informatik
DOI: 10.4230/lipics.icalp.2020.7

Peer reviewed articles (1)

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Author(s): Shigang Li, Tal Ben-Nun, Giorgi Nadiradze, Salvatore Digirolamo, Nikoli Dryden, Dan Alistarh, Torsten Hoefler
Published in: IEEE Transactions on Parallel and Distributed Systems, 2020, Page(s) 1-1, ISSN 1045-9219
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpds.2020.3040606