Statistical Mechanics of Learning

Project Information

SMILE

Grant agreement ID: 714608

DOI

10.3030/714608

Project closed

EC signature date 16 November 2016

Start date 1 September 2017

End date 28 February 2023

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 346 795,00

EU contribution

€ 1 346 795,00

Coordinated by

ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE
Switzerland

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Publications

Peer reviewed articles (32)

The planted XY model: thermodynamics and inference

Author(s): Siyu Chen; Guanhao Huang; Giovanni Piccioli; Lenka Zdeborová
Published in: Phys. Rev. E, Issue 106, 2022, Page(s) 054115, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.48550/arxiv.2208.06488

Probing transfer learning with a model of synthetic correlated datasets

Author(s): Federica Gerace; Luca Saglietti; Stefano Sarao Mannelli; Andrew Saxe; Lenka Zdeborová
Published in: Mach. Learn.: Sci. Technol., Issue 5, 2022, Page(s) 015030, ISSN 2632-2153
Publisher: IOP publishing
DOI: 10.1088/2632-2153/ac4f3f

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

Author(s): Antoine Maillard; Florent Krzakala; Marc Mézard; and Lenka Zdeborová;
Published in: J. Stat. Mech., Issue 2022, 2022, Page(s) 083301, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac7e4c

Bayesian reconstruction of memories stored in neural networks from their connectivity

Author(s): Sebastian Goldt; Florent Krzakala; Lenka Zdeborová; Nicolas Brunel
Published in: PLoS Comput Biol, Issue 19, 2023, Page(s) 1, ISSN 1553-734X
Publisher: Public Library of Science
DOI: 10.48550/arxiv.2105.07416

Learning curves of generic features maps for realistic datasets with a teacher-student model

Author(s): Loureiro, Bruno; Gerbelot, Cédric; Cui, Hugo; Goldt, Sebastian; Krzakala, Florent; Mézard, Marc; Zdeborová, Lenka
Published in: J. Stat. Mech., Issue 114001, 2022, Page(s) 114001, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac9825

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Author(s): Mignacco, Francesca; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Journal of Statistical Mechanics: Theory and Experiment,, 2021, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac3a80

Modeling the Influence of Data Structure on Learning in Neural Networks: The Hidden Manifold Model

Author(s): Sebastian Goldt; Marc Mézard; Florent Krzakala; Lenka Zdeborová
Published in: PHYSICAL REVIEW X, Issue 10, 041044, 2020, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/physrevx.10.041044

Large Deviations of Semi-supervised Learning in the Stochastic Block Model

Author(s): Cui, Hugo; Saglietti, Luca; Zdeborová, Lenka
Published in: Phys. Rev. E, Issue 105, 034108, 2022, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/physreve.105.034108

Large deviations for the perceptron model and consequences for active learning

Author(s): Cui, Hugo; Saglietti, Luca; Zdeborová, Lenka
Published in: Machine Learning: Science and Technology, Issue Volume 2, Number 4, 2021, ISSN 2632-2153
Publisher: IOP Publishing Ltd
DOI: 10.1088/2632-2153/abfbbb

Generalisation error in learning with random features and the hidden manifold model*

Author(s): Gerace, Federica; Loureiro, Bruno; Krzakala, Florent; Mezard, Marc; Zdeborová, Lenka
Published in: Journal of Statistical Mechanics: Theory and Experiment, 2021, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac3ae6

Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup.

Author(s): Sebastian Goldt; Madhu Advani; Andrew M. Saxe; Florent Krzakala; Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, 2020, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/abc61e

Glassy Nature of the Hard Phase in Inference Problems

Author(s): Fabrizio Antenucci, Silvio Franz, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Physical Review X, Issue 9/1, 2019, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/PhysRevX.9.011020

Optimal errors and phase transitions in high-dimensional generalized linear models

Author(s): Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, Lenka Zdeborová
Published in: Proceedings of the National Academy of Sciences, Issue 116/12, 2019, Page(s) 5451-5460, ISSN 0027-8424
Publisher: National Academy of Sciences
DOI: 10.1073/pnas.1802705116

Typology of phase transitions in Bayesian inference problems

Author(s): Federico Ricci-Tersenghi, Guilhem Semerjian, Lenka Zdeborová
Published in: Physical Review E, Issue 99/4, 2019, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/PhysRevE.99.042109

Approximate survey propagation for statistical inference

Author(s): Fabrizio Antenucci, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/2, 2019, Page(s) 023401, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/aafa7d

Information-theoretic thresholds from the cavity method

Author(s): Amin Coja-Oghlan, Florent Krzakala, Will Perkins, Lenka Zdeborová
Published in: Advances in Mathematics, Issue 333, 2018, Page(s) 694-795, ISSN 0001-8708
Publisher: Academic Press
DOI: 10.1016/j.aim.2018.05.029

Minimal sets to destroy the k -core in random networks

Author(s): Christian Schmidt, Henry D. Pfister, Lenka Zdeborová
Published in: Physical Review E, Issue 99/2, 2019, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/PhysRevE.99.022310

Storage capacity in symmetric binary perceptrons

Author(s): Benjamin Aubin, Will Perkins, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 52/29, 2019, Page(s) 294003, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab227a

Machine learning and the physical sciences

Author(s): Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, Lenka Zdeborová
Published in: Reviews of Modern Physics, Issue 91/4, 2019, ISSN 0034-6861
Publisher: American Physical Society
DOI: 10.1103/RevModPhys.91.045002

On the universality of noiseless linear estimation with respect to the measurement matrix

Author(s): Alia Abbara, Antoine Baker, Florent Krzakala, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 53/16, 2020, Page(s) 164001, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab59ef

Entropy and mutual information in models of deep neural networks

Author(s): Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/12, 2019, Page(s) 124014, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab3430

The committee machine: computational to statistical gaps in learning a two-layers neural network

Author(s): Benjamin Aubin, Antoine Maillard, Jean Barbier, Florent Krzakala, Nicolas Macris, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/12, 2019, Page(s) 124023, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab43d2

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-Dimensional Inference

Author(s): Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Physical Review X, Issue 10/1, 2020, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/physrevx.10.011057

Dense limit of the Dawid–Skene model for crowdsourcing and regions of sub-optimality of message passing algorithms

Author(s): Christian Schmidt, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 53/12, 2020, Page(s) 124001, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab757f

Thresholds of descending algorithms in inference problems

Author(s): Stefano Sarao Mannelli, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2020/3, 2020, Page(s) 034004, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab7123

Blind calibration for compressed sensing: State evolution and an online algorithm

Author(s): Marylou Gabrié, Jean Barbier, Florent Krzakala, Lenka Zdeborova
Published in: Journal of Physics A: Mathematical and Theoretical, 2020, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab8416

High-temperature expansions and message passing algorithms

Author(s): Antoine Maillard, Laura Foini, Alejandro Lage Castellanos, Florent Krzakala, Marc Mézard, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/11, 2019, Page(s) 113301, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab4bbb

Learning curves for the multi-class teacher-student perceptron

Author(s): Elisabetta Cornacchia; Francesca Mignacco; Rodrigo Veiga; Cedric Gerbelot; Bruno Loureiro; Lenka Zdeborova
Published in: Mach. Learn.: Sci. Technol., Issue 4, 2023, Page(s) 015019, ISSN 2632-2153
Publisher: Institute of Physics
DOI: 10.48550/arxiv.2203.12094

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Author(s): Francesca Mignacco; Pierfrancesco Urbani; Lenka Zdeborová
Published in: Machine Learning: Science and Technology, Issue 2 (3), 2021, Page(s) 035029, ISSN 2632-2153
Publisher: IOP Publishing
DOI: 10.1088/2632-2153/ac0615

(Dis)assortative partitions on random regular graphs

Author(s): Behrens, Freya; Arpino, Gabriel; Kivva, Yaroslav; Zdeborová, Lenka
Published in: J. Phys. A: Math. Theor., Issue 55, 2022, Page(s) 395004, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ac8b46

An analytical theory of curriculum learning in teacher–student networks*

Author(s): Luca Saglietti; Stefano Sarao Mannelli; Andrew Saxe
Published in: J. Stat. Mech., Issue 2022, 2022, Page(s) 114014, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac9b3c

The Spiked Matrix Model With Generative Priors

Author(s): Benjamin Aubin; Bruno Loureiro; Antoine Maillard; Florent Krzakala; Lenka Zdeborová
Published in: IEEE Transactions on Information Theory, Issue Volume: 67, Issue: 2, 2021, ISSN 0018-9448
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tit.2020.3033985

Conference proceedings (23)

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Author(s): Aubin, Benjamin; Krzakala, Florent; Lu, Yue M.; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Multi-layer State Evolution Under Random Convolutional Design

Author(s): Max Daniels, Cedric Gerbelot, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NuerIPS
DOI: 10.48550/arxiv.2205.13503

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Author(s): Bruno Loureiro, Gabriele Sicuro, Cedric Gerbelot, Alessandro Pacco, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

The role of regularization in classification of high-dimensional noisy Gaussian mixture

Author(s): Mignacco, Francesca; Krzakala, Florent; Lu, Yue,; Zdeborová, Lenka
Published in: Proceedings of the 37th International Conference on Machine Learning, Issue PMLR 119:6874-6883, 2020
Publisher: Cambridge MA: JMLR

Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning

Author(s): Abbara, Alia; Aubin, Benjamin; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of The First Mathematical and Scientific Machine Learning Conference,, Issue PMLR 107, 2020, Page(s) 27-54, ISSN 2640-3498
Publisher: Cambridge MA: JMLR

Solvable Model for Inheriting the Regularization through Knowledge Distillation

Author(s): Saglietti, Luca; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 809-846
Publisher: PMLR
DOI: 10.48550/arxiv.2012.00194

Exact asymptotics for phase retrieval and compressed sensing with random generative priors

Author(s): Aubin, Benjamin; Loureiro, Bruno; Baker, Antoine; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of The First Mathematical and Scientific Machine Learning Conference,, Issue PMLR 107, 2020, Page(s) 55-73
Publisher: Cambridge MA: JMLR

The Gaussian equivalence of generative models for learning with two-layer neural networks

Author(s): Goldt, Sebastian; Loureiro, Bruno; Reeves, Galen; Krzakala, Florent; Mezard, Marc; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 426-471
Publisher: PMLR
DOI: 10.48550/arxiv.2006.14709

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Author(s): Refinetti, Maria; Goldt, Sebastian; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of the 38th International Conference on Machine Learning, Issue PMLR 139:8936-8947, 2021
Publisher: Cambridge MA: JMLR

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Author(s): Cui, Hugo; Loureiro, Bruno; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

Streaming Bayesian inference: Theoretical limits and mini-batch approximate message-passing

Author(s): Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborova
Published in: 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2017, Page(s) 1048-1055, ISBN 978-1-5386-3266-6
Publisher: IEEE
DOI: 10.1109/allerton.2017.8262853

Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model

Author(s): Mannelli, Stefano Sarao; Biroli, Giulio; Cammarota, Chiara; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 32, 2019
Publisher: Curran Associates, Inc.

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

Author(s): Mannelli, Stefano Sarao; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Proceedings of Machine Learning Research, 2019
Publisher: PMLR

Generalisation dynamics of online learning in over-parameterised neural networks

Author(s): Goldt, Sebastian; Advani, Madhu S.; Saxe, Andrew M.; Krzakala, Florent; Zdeborová, Lenka
Published in: https://hal-cea.archives-ouvertes.fr/cea-02009764, Issue 1, 2019
Publisher: N/A

The spiked matrix model with generative priors

Author(s): Benjamin Aubin, Bruno Loureiro, Antoine Maillard, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems 32, 2019
Publisher: Curran Associates, Inc.

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Author(s): Stefano, Sarao Mannelli; Biroli, Giulio; Cammarota, Chiara; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Construction of optimal spectral methods in phase retrieval

Author(s): Maillard, Antoine; Krzakala, Florent; Yue, lu; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 693-72
Publisher: PMLR
DOI: 10.48550/arxiv.2012.04524

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Author(s): Luca Pesce, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NeurIPS
DOI: 10.48550/arxiv.2205.13527

Phase retrieval in high dimensions: Statistical and computational phase transitions

Author(s): Maillard, Antoine; Loureiro, Bruno; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Author(s): Sarao Manelli, Stefano; Vanden-Eijnden, Eric; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Learning curves of generic features maps for realistic datasets with a teacher-student model

Author(s): Loureiro, Bruno; Gerbelot, Cédric; Cui, Hugo; Goldt, Sebastian; Krzakala, Florent; Mézard, Marc; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Author(s): Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NeurIPS
DOI: 10.48550/arxiv.2202.00293

Optimal denoising of rotationally invariant rectangular matrices

Author(s): Emanuele Troiani, Vittorio Erba, Florent Krzakala, Antoine Maillard, Lenka Zdeborova
Published in: Proceedings of Mathematical and Scientific Machine Learning, Issue 190, 2022, Page(s) 97-112
Publisher: PMLR
DOI: 10.48550/arxiv.2203.07752