Dynamics-Aware Theory of Deep Learning

Project Information

DYNASTY

Grant agreement ID: 101039676

DOI

10.3030/101039676

EC signature date 3 June 2022

Start date 1 October 2022

End date 30 September 2027

Funded under

• European Research Council (ERC)

Total cost

€ 1 498 410,00

EU contribution

€ 1 498 410,00

Coordinated by

INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE
France

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Documents, reports (1)

Data Management Plan

Drafting of Data Management Plan

Publications

Conference proceedings (15)

Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD

Author(s): Wan, Yijun; Barsbey, Melih; Şimşekli, Umut; Zaidi, Abdellatif
Published in: The Thirty-Seventh Annual Conference on Neural Information Processing Systems (NeurIPS), Dec 2023, New Orleans, United States, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2306.08125

Generalization Bounds for Heavy-Tailed SDEs through the Fractional Fokker-Planck Equation

Author(s): Benjamin Dupuis, Umut Simsekli
Published in: NeurIPS, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2402.07723

Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent

Author(s): Krunoslav Pavasovic, Alain Durmus, Umut Simsekli
Published in: NeurIPS, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2310.18455

From Mutual Information to Expected Dynamics: New Generalization Bounds for Heavy-Tailed SGD

Author(s): Benjamin Dupuis, Paul Viallard
Published in: NeurIPS Workshop on Heavy Tails in Machine Learning, 2023
Publisher: Openreview

Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent

Author(s): Lim, Soon Hoe; Wan, Yijun; Şimşekli, Umut
Published in: NeurIPS 2022 - Thirty-sixth Conference on Neural Information Processing Systems, 2022
Publisher: PMLR
DOI: 10.48550/arxiv.2205.11361

Piecewise deterministic generative models

Author(s): Andrea Bertazzi, Dario Shariatian, Umut Simsekli, Eric Moulines, Alain Durmus
Published in: NeurIPS, 2024
Publisher: PMLR
DOI: 10.48550/arxiv.2407.19448

Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions

Author(s): Raj, Anant; Zhu, Lingjiong; Gürbüzbalaban, Mert; Şimşekli, Umut
Published in: ICML, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2301.11885

Learning via Wasserstein-Based High Probability Generalisation Bounds

Author(s): Viallard, Paul; Haddouche, Maxime; Şimşekli, Umut; Guedj, Benjamin
Published in: NeurIPS 2023, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2306.04375

Generalization Bounds with Data-dependent Fractal Dimensions

Author(s): Dupuis, Benjamin; Deligiannidis, George; Şimşekli, Umut
Published in: ICML, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2302.02766

Generalization Guarantees via Algorithm-dependent Rademacher Complexity

Author(s): Sachs, Sarah; van Erven, Tim; Hodgkinson, Liam; Khanna, Rajiv; Şimşekli, Umut
Published in: COLT 2023 - 36th Annual Conference on Learning Theory, 2023, Bangalore (Virtual event), India, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2307.02501

Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms

Author(s): Rayna Andreeva, Benjamin Dupuis, Rik Sarkar, Tolga Birdal, Umut Simsekli
Published in: NeurIPS, 2024
Publisher: PMLR
DOI: 10.48550/arxiv.2407.08723

Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent

Author(s): Zhu, Lingjiong; Gurbuzbalaban, Mert; Raj, Anant; Şimşekli, Umut
Published in: NeurIPS 2023 - Thirty-seventh Annual Conference on Neural Information Processing Systems,, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2305.12056

Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers

Author(s): Park, Sejun; Şimşekli, Umut; Erdogdu, Murat A.
Published in: NeurIPS 2022 - Thirty-sixth Conference on Neural Information Processing Systems, 2022, ISSN 1049-5258
Publisher: PMLR
DOI: 10.48550/arxiv.2209.08951

Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models

Author(s): Raj, Anant; Şimşekli, Umut; Rudi, Alessandro
Published in: NeurIPS 2023 - Thirty-seventh Annual Conference on Neural Information Processing Systems, Dec 2023, New Orleans, United States, 2023
Publisher: PMLR
DOI: 10.48550/arxiv.2303.17109

Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares

Author(s): Raj, Anant; Barsbey, Melih; Gürbüzbalaban, Mert; Zhu, Lingjiong; Şimşekli, Umut
Published in: ALT 2023 - 34th International Conference on Algorithmic Learning Theory, Feb 2023, Singapore, Singapore, 2022
Publisher: PMLR
DOI: 10.48550/arxiv.2206.01274

Peer reviewed articles (1)

Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize

Author(s): Gürbüzbalaban, Mert; Hu, Yuanhan; Şimşekli, Umut; Zhu, Lingjiong
Published in: TMLR, 2023, ISSN 2835-8856
Publisher: PMLR
DOI: 10.48550/arxiv.2302.05516