Statistical Mechanics of Learning

Publications

The planted XY model: thermodynamics and inference

Author(s): Siyu Chen; Guanhao Huang; Giovanni Piccioli; Lenka Zdeborová
Published in: Phys. Rev. E, Issue 106, 2022, Page(s) 054115, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.48550/arxiv.2208.06488

Probing transfer learning with a model of synthetic correlated datasets

Author(s): Federica Gerace; Luca Saglietti; Stefano Sarao Mannelli; Andrew Saxe; Lenka Zdeborová
Published in: Mach. Learn.: Sci. Technol., Issue 5, 2022, Page(s) 015030, ISSN 2632-2153
Publisher: IOP publishing
DOI: 10.1088/2632-2153/ac4f3f

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

Author(s): Antoine Maillard; Florent Krzakala; Marc Mézard; and Lenka Zdeborová;
Published in: J. Stat. Mech., Issue 2022, 2022, Page(s) 083301, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac7e4c

Bayesian reconstruction of memories stored in neural networks from their connectivity

Author(s): Sebastian Goldt; Florent Krzakala; Lenka Zdeborová; Nicolas Brunel
Published in: PLoS Comput Biol, Issue 19, 2023, Page(s) 1, ISSN 1553-734X
Publisher: Public Library of Science
DOI: 10.48550/arxiv.2105.07416

Learning curves of generic features maps for realistic datasets with a teacher-student model

Author(s): Loureiro, Bruno; Gerbelot, Cédric; Cui, Hugo; Goldt, Sebastian; Krzakala, Florent; Mézard, Marc; Zdeborová, Lenka
Published in: J. Stat. Mech., Issue 114001, 2022, Page(s) 114001, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac9825

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*

Author(s): Mignacco, Francesca; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Journal of Statistical Mechanics: Theory and Experiment,, 2021, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac3a80

Modeling the Influence of Data Structure on Learning in Neural Networks: The Hidden Manifold Model

Author(s): Sebastian Goldt; Marc Mézard; Florent Krzakala; Lenka Zdeborová
Published in: PHYSICAL REVIEW X, Issue 10, 041044, 2020, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/physrevx.10.041044

Large Deviations of Semi-supervised Learning in the Stochastic Block Model

Author(s): Cui, Hugo; Saglietti, Luca; Zdeborová, Lenka
Published in: Phys. Rev. E, Issue 105, 034108, 2022, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/physreve.105.034108

Large deviations for the perceptron model and consequences for active learning

Author(s): Cui, Hugo; Saglietti, Luca; Zdeborová, Lenka
Published in: Machine Learning: Science and Technology, Issue Volume 2, Number 4, 2021, ISSN 2632-2153
Publisher: IOP Publishing Ltd
DOI: 10.1088/2632-2153/abfbbb

Generalisation error in learning with random features and the hidden manifold model*

Author(s): Gerace, Federica; Loureiro, Bruno; Krzakala, Florent; Mezard, Marc; Zdeborová, Lenka
Published in: Journal of Statistical Mechanics: Theory and Experiment, 2021, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac3ae6

Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup.

Author(s): Sebastian Goldt; Madhu Advani; Andrew M. Saxe; Florent Krzakala; Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, 2020, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/abc61e

Glassy Nature of the Hard Phase in Inference Problems

Author(s): Fabrizio Antenucci, Silvio Franz, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Physical Review X, Issue 9/1, 2019, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/PhysRevX.9.011020

Optimal errors and phase transitions in high-dimensional generalized linear models

Author(s): Jean Barbier, Florent Krzakala, Nicolas Macris, Léo Miolane, Lenka Zdeborová
Published in: Proceedings of the National Academy of Sciences, Issue 116/12, 2019, Page(s) 5451-5460, ISSN 0027-8424
Publisher: National Academy of Sciences
DOI: 10.1073/pnas.1802705116

Typology of phase transitions in Bayesian inference problems

Author(s): Federico Ricci-Tersenghi, Guilhem Semerjian, Lenka Zdeborová
Published in: Physical Review E, Issue 99/4, 2019, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/PhysRevE.99.042109

Approximate survey propagation for statistical inference

Author(s): Fabrizio Antenucci, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/2, 2019, Page(s) 023401, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/aafa7d

Information-theoretic thresholds from the cavity method

Author(s): Amin Coja-Oghlan, Florent Krzakala, Will Perkins, Lenka Zdeborová
Published in: Advances in Mathematics, Issue 333, 2018, Page(s) 694-795, ISSN 0001-8708
Publisher: Academic Press
DOI: 10.1016/j.aim.2018.05.029

Minimal sets to destroy the k -core in random networks

Author(s): Christian Schmidt, Henry D. Pfister, Lenka Zdeborová
Published in: Physical Review E, Issue 99/2, 2019, ISSN 2470-0045
Publisher: American Physical Society
DOI: 10.1103/PhysRevE.99.022310

Storage capacity in symmetric binary perceptrons

Author(s): Benjamin Aubin, Will Perkins, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 52/29, 2019, Page(s) 294003, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab227a

Machine learning and the physical sciences

Author(s): Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, Lenka Zdeborová
Published in: Reviews of Modern Physics, Issue 91/4, 2019, ISSN 0034-6861
Publisher: American Physical Society
DOI: 10.1103/RevModPhys.91.045002

On the universality of noiseless linear estimation with respect to the measurement matrix

Author(s): Alia Abbara, Antoine Baker, Florent Krzakala, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 53/16, 2020, Page(s) 164001, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab59ef

Entropy and mutual information in models of deep neural networks

Author(s): Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/12, 2019, Page(s) 124014, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab3430

The committee machine: computational to statistical gaps in learning a two-layers neural network

Author(s): Benjamin Aubin, Antoine Maillard, Jean Barbier, Florent Krzakala, Nicolas Macris, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/12, 2019, Page(s) 124023, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab43d2

Marvels and Pitfalls of the Langevin Algorithm in Noisy High-Dimensional Inference

Author(s): Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Published in: Physical Review X, Issue 10/1, 2020, ISSN 2160-3308
Publisher: American Physical Society
DOI: 10.1103/physrevx.10.011057

Dense limit of the Dawid–Skene model for crowdsourcing and regions of sub-optimality of message passing algorithms

Author(s): Christian Schmidt, Lenka Zdeborová
Published in: Journal of Physics A: Mathematical and Theoretical, Issue 53/12, 2020, Page(s) 124001, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab757f

Thresholds of descending algorithms in inference problems

Author(s): Stefano Sarao Mannelli, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2020/3, 2020, Page(s) 034004, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab7123

Blind calibration for compressed sensing: State evolution and an online algorithm

Author(s): Marylou Gabrié, Jean Barbier, Florent Krzakala, Lenka Zdeborova
Published in: Journal of Physics A: Mathematical and Theoretical, 2020, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ab8416

High-temperature expansions and message passing algorithms

Author(s): Antoine Maillard, Laura Foini, Alejandro Lage Castellanos, Florent Krzakala, Marc Mézard, Lenka Zdeborová
Published in: Journal of Statistical Mechanics: Theory and Experiment, Issue 2019/11, 2019, Page(s) 113301, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ab4bbb

Learning curves for the multi-class teacher-student perceptron

Author(s): Elisabetta Cornacchia; Francesca Mignacco; Rodrigo Veiga; Cedric Gerbelot; Bruno Loureiro; Lenka Zdeborova
Published in: Mach. Learn.: Sci. Technol., Issue 4, 2023, Page(s) 015019, ISSN 2632-2153
Publisher: Institute of Physics
DOI: 10.48550/arxiv.2203.12094

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

Author(s): Francesca Mignacco; Pierfrancesco Urbani; Lenka Zdeborová
Published in: Machine Learning: Science and Technology, Issue 2 (3), 2021, Page(s) 035029, ISSN 2632-2153
Publisher: IOP Publishing
DOI: 10.1088/2632-2153/ac0615

(Dis)assortative partitions on random regular graphs

Author(s): Behrens, Freya; Arpino, Gabriel; Kivva, Yaroslav; Zdeborová, Lenka
Published in: J. Phys. A: Math. Theor., Issue 55, 2022, Page(s) 395004, ISSN 1751-8113
Publisher: Institute of Physics Publishing
DOI: 10.1088/1751-8121/ac8b46

An analytical theory of curriculum learning in teacher–student networks*

Author(s): Luca Saglietti; Stefano Sarao Mannelli; Andrew Saxe
Published in: J. Stat. Mech., Issue 2022, 2022, Page(s) 114014, ISSN 1742-5468
Publisher: Institute of Physics
DOI: 10.1088/1742-5468/ac9b3c

The Spiked Matrix Model With Generative Priors

Author(s): Benjamin Aubin; Bruno Loureiro; Antoine Maillard; Florent Krzakala; Lenka Zdeborová
Published in: IEEE Transactions on Information Theory, Issue Volume: 67, Issue: 2, 2021, ISSN 0018-9448
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tit.2020.3033985

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Author(s): Aubin, Benjamin; Krzakala, Florent; Lu, Yue M.; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Multi-layer State Evolution Under Random Convolutional Design

Author(s): Max Daniels, Cedric Gerbelot, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NuerIPS
DOI: 10.48550/arxiv.2205.13503

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Author(s): Bruno Loureiro, Gabriele Sicuro, Cedric Gerbelot, Alessandro Pacco, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

The role of regularization in classification of high-dimensional noisy Gaussian mixture

Author(s): Mignacco, Francesca; Krzakala, Florent; Lu, Yue,; Zdeborová, Lenka
Published in: Proceedings of the 37th International Conference on Machine Learning, Issue PMLR 119:6874-6883, 2020
Publisher: Cambridge MA: JMLR

Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning

Author(s): Abbara, Alia; Aubin, Benjamin; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of The First Mathematical and Scientific Machine Learning Conference,, Issue PMLR 107, 2020, Page(s) 27-54, ISSN 2640-3498
Publisher: Cambridge MA: JMLR

Solvable Model for Inheriting the Regularization through Knowledge Distillation

Author(s): Saglietti, Luca; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 809-846
Publisher: PMLR
DOI: 10.48550/arxiv.2012.00194

Exact asymptotics for phase retrieval and compressed sensing with random generative priors

Author(s): Aubin, Benjamin; Loureiro, Bruno; Baker, Antoine; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of The First Mathematical and Scientific Machine Learning Conference,, Issue PMLR 107, 2020, Page(s) 55-73
Publisher: Cambridge MA: JMLR

The Gaussian equivalence of generative models for learning with two-layer neural networks

Author(s): Goldt, Sebastian; Loureiro, Bruno; Reeves, Galen; Krzakala, Florent; Mezard, Marc; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 426-471
Publisher: PMLR
DOI: 10.48550/arxiv.2006.14709

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Author(s): Refinetti, Maria; Goldt, Sebastian; Krzakala, Florent; Zdeborová, Lenka
Published in: Proceedings of the 38th International Conference on Machine Learning, Issue PMLR 139:8936-8947, 2021
Publisher: Cambridge MA: JMLR

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Author(s): Cui, Hugo; Loureiro, Bruno; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

Streaming Bayesian inference: Theoretical limits and mini-batch approximate message-passing

Author(s): Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborova
Published in: 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2017, Page(s) 1048-1055, ISBN 978-1-5386-3266-6
Publisher: IEEE
DOI: 10.1109/allerton.2017.8262853

Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model

Author(s): Mannelli, Stefano Sarao; Biroli, Giulio; Cammarota, Chiara; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 32, 2019
Publisher: Curran Associates, Inc.

Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

Author(s): Mannelli, Stefano Sarao; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Proceedings of Machine Learning Research, 2019
Publisher: PMLR

Generalisation dynamics of online learning in over-parameterised neural networks

Author(s): Goldt, Sebastian; Advani, Madhu S.; Saxe, Andrew M.; Krzakala, Florent; Zdeborová, Lenka
Published in: https://hal-cea.archives-ouvertes.fr/cea-02009764, Issue 1, 2019
Publisher: N/A

The spiked matrix model with generative priors

Author(s): Benjamin Aubin, Bruno Loureiro, Antoine Maillard, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems 32, 2019
Publisher: Curran Associates, Inc.

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Author(s): Stefano, Sarao Mannelli; Biroli, Giulio; Cammarota, Chiara; Krzakala, Florent; Urbani, Pierfrancesco; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Construction of optimal spectral methods in phase retrieval

Author(s): Maillard, Antoine; Krzakala, Florent; Yue, lu; Zdeborová, Lenka
Published in: Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Issue 145, 2022, Page(s) 693-72
Publisher: PMLR
DOI: 10.48550/arxiv.2012.04524

Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap

Author(s): Luca Pesce, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NeurIPS
DOI: 10.48550/arxiv.2205.13527

Phase retrieval in high dimensions: Statistical and computational phase transitions

Author(s): Maillard, Antoine; Loureiro, Bruno; Krzakala, Florent; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Author(s): Sarao Manelli, Stefano; Vanden-Eijnden, Eric; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 2020
Publisher: NeurIPS Proceedings

Learning curves of generic features maps for realistic datasets with a teacher-student model

Author(s): Loureiro, Bruno; Gerbelot, Cédric; Cui, Hugo; Goldt, Sebastian; Krzakala, Florent; Mézard, Marc; Zdeborová, Lenka
Published in: Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021
Publisher: NeurIPS Proceedings

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Author(s): Rodrigo Veiga, Ludovic Stephan, Bruno Loureiro, Florent Krzakala, Lenka Zdeborová
Published in: Advances in Neural Information Processing Systems (NeurIPS 2022), Issue 35, 2022
Publisher: NeurIPS
DOI: 10.48550/arxiv.2202.00293

Optimal denoising of rotationally invariant rectangular matrices

Author(s): Emanuele Troiani, Vittorio Erba, Florent Krzakala, Antoine Maillard, Lenka Zdeborova
Published in: Proceedings of Mathematical and Scientific Machine Learning, Issue 190, 2022, Page(s) 97-112
Publisher: PMLR
DOI: 10.48550/arxiv.2203.07752

Searching for OpenAIRE data...

Publications

Share this page

Download