Elastic Coordination for Scalable Machine Learning

Informazioni relative al progetto

ScaleML

ID dell’accordo di sovvenzione: 805223

DOI

10.3030/805223

Progetto chiuso

Data della firma CE 17 Ottobre 2018

Data di avvio 1 Marzo 2019

Data di completamento 29 Febbraio 2024

Finanziato da

• EXCELLENT SCIENCE - European Research Council (ERC)

Costo totale

€ 1 494 121,00

Contributo UE

€ 1 494 121,00

Coordinato da

INSTITUTE OF SCIENCE AND TECHNOLOGY AUSTRIA
Austria

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Pubblicazioni

Conference proceedings (18)

On the sample complexity of adversarial multi-source PAC learning

Autori: Nikola Konstantinov, Elias Frantar, Dan Alistarh, Christoph H. Lampert
Pubblicato in: Proceedings of the 37th International Conference on Machine Learning (ICML 2020), 2020
Editore: PMLR

Taming unbalanced training workloads in deep learning with partial collective operations

Autori: Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo, Dan Alistarh, Torsten Hoefler
Pubblicato in: Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2020, Pagina/e 45-61, ISBN 9781450368186
Editore: ACM
DOI: 10.1145/3332466.3374528

The splay-list: a distribution-adaptive concurrent skip-list

Autori: Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova, Amirkeivan Mohtashami
Pubblicato in: Proceedings of the 34th International Symposium on Distributed Computing (DISC 2020), 2020
Editore: LIPIcs
DOI: 10.4230/lipics.disc.2020.3

Non-blocking interpolation search trees with doubly-logarithmic running time

Autori: Trevor Brown, Aleksandar Prokopec, Dan Alistarh
Pubblicato in: Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2020, Pagina/e 276-291, ISBN 9781450368186
Editore: ACM
DOI: 10.1145/3332466.3374542

AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks

Autori: Alexandra Peste, Eugenia Iofinova, Adrian Vladu, Dan Alistarh
Pubblicato in: NeurIPS Proceedings, 2021
Editore: NeurIPS

Towards Tight Communication Lower Bounds for Distributed Optimisation

Autori: Janne Korhonen, Dan Alistarh
Pubblicato in: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021
Editore: NeurIPS

Efficiency guarantees for parallel incremental algorithms under relaxed schedulers

Autori: Dan Alistarh, Nikita Koval, Giorgi Nadiradze
Pubblicato in: 31st ACM Symposium on Parallelism in Algorithms and Architectures, 2019, Pagina/e 145-154, ISBN 9781450361842
Editore: ACM
DOI: 10.1145/3323165.3323201

Byzantine-Resilient Non-Convex Stochastic Gradient Descent

Autori: Zeyuan Allen-Zhu, Faeze Ebrahimianghazani, Jerry Li, Dan Alistarh
Pubblicato in: Proceedings of the International Conference on Learning Representations (ICLR), 2021
Editore: Openreview

Powerset convolutional neural networks

Autori: Chris Wendler, Dan Alistarh, Markus Püschel
Pubblicato in: Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), Numero 32, 2019, Pagina/e 927-938
Editore: NIPS: Conference on Neural Information Processing Systems

WoodFisher: Efficient Second-Order Approximation for Neural Network Compression

Autori: Sidak Pal Singh, Dan Alistarh
Pubblicato in: Proceedings of the International Conference on Neural Information Processing Systems, 2020
Editore: NeurIPS

Communication-Efficient Federated Learning With Data and Client Heterogeneity

Autori: Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze, Dan Alistarh
Pubblicato in: Proceedings of AISTATS 2024, 2024
Editore: PMLR

Asynchronous Decentralized SGD with Quantized and Local Updates

Autori: Giorgi Nadiradze, Amirmojtaba Sabour, Peter Davies, Shigang Li, Dan Alistarh
Pubblicato in: NeurIPS Proceedings, 2021
Editore: NeurIPS Proceedings

Lower Bounds for Shared-Memory Leader Election under Bounded Write Contention

Autori: Dan Alistarh, Rati Gelashvili, Giorgi Nadiradze
Pubblicato in: Proceedings of the 35th International Symposium on Distributed Computing (DISC 2021), 2021, Pagina/e 4:1--4:17
Editore: "Schloss Dagstuhl -- Leibniz-Zentrum f{\""u}r Informatik"
DOI: 10.4230/lipics.disc.2021.4

Relaxed Scheduling for Scalable Belief Propagation

Autori: Vitaly Aksenov, Janne Korhonen, Dan Alistarh
Pubblicato in: Proceedings of the 36th International Conference on Neural Information Processing Systems, 2020
Editore: NeurIPS

L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learning

Autori: Mohammadreza Alimohammadi, Ilia Markov, Elias Frantar, Dan Alistarh
Pubblicato in: Proceedings of MLSys 2024, 2024
Editore: MLSys

SparCML - high-performance sparse communication for machine learning

Autori: Cedric Renggli, Saleh Ashkboos, Mehdi Aghagolzadeh, Dan Alistarh, Torsten Hoefler
Pubblicato in: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2019, Pagina/e 1-15, ISBN 9781450362290
Editore: ACM
DOI: 10.1145/3295500.3356222

New Bounds For Distributed Mean Estimation and Variance Reduction

Autori: Peter Davies, Vijaykrishna Gurunathan, Niusha Moshrefi, Saleh Ashkboos, Dan Alistarh
Pubblicato in: Proceedings of the International Conference on Learning Representations, 2021
Editore: Openreview

Dynamic averaging load balancing on cycles

Autori: Dan Alistarh, Giorgi Nadiradze, Amirmojtaba Sabour
Pubblicato in: Leibniz International Proceedings in Informatics, Numero 168/7, 2020, ISBN 9783959771382
Editore: Schloss Dagstuhl - Leibniz-Zentrum für Informatik
DOI: 10.4230/lipics.icalp.2020.7

Peer reviewed articles (1)

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Autori: Shigang Li, Tal Ben-Nun, Giorgi Nadiradze, Salvatore Digirolamo, Nikoli Dryden, Dan Alistarh, Torsten Hoefler
Pubblicato in: IEEE Transactions on Parallel and Distributed Systems, 2020, Pagina/e 1-1, ISSN 1045-9219
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tpds.2020.3040606