A Theory for Understanding, Designing, and Training Deep Learning Systems

Neural Networks with Small Weights and Depth-Separation Barriers

Author(s): Gal Vardi and Ohad Shamir
Published in: 2020
Publisher: NeurIPS 2020

On Margin Maximization in Linear and ReLU Networks

Author(s): Gal Vardi, Gilad Yehudai, Ohad Shamir
Published in: 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Issue 36, 2022
Publisher: NeurIPS

Initialization-dependent sample complexity of linear predictors and neural networks

Author(s): R Magen, O Shamir
Published in: 2023
Publisher: Advances in Neural Information Processing Systems

Implicit Regularization Towards Rank Minimization in ReLU Networks

Author(s): Nadav Timor, Gal Vardi and Ohad Shamir
Published in: 2023
Publisher: ALT

The Sample Complexity of One-Hidden-Layer Neural Networks

Author(s): Gal Vardi, Ohad Shamir and Nathan Srebro
Published in: 2022
Publisher: NeurIPS

Learning a Single Neuron with Bias Using Gradient Descent

Author(s): Gal Vardi, Gilad Yehudai, Ohad Shamir
Published in: 2021
Publisher: NeurIP

Deterministic nonsmooth nonconvex optimization

Author(s): M Jordan, G Kornowski, T Lin, O Shamir, M Zampetakis
Published in: 2023
Publisher: Proceedings of COLT 2023

Width is Less Important than Depth in ReLU Neural Networks

Author(s): Gal Vardi, Gilad Yehudai and Ohad Shamir
Published in: 2022
Publisher: COLT

Open Problem: Anytime Convergence Rate of Gradient Descent

Author(s): G Kornowski, O Shamir
Published in: 2024
Publisher: Proceedings of COLT 2024

Gradient Methods Provably Converge to Non-Robust Networks

Author(s): Gal Vardi, Gilad Yehudai and Ohad Shamir
Published in: 2022
Publisher: NeurIPS

Reconstructing Training Data from Trained Neural Networks

Author(s): Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, Michal Irani
Published in: 2022
Publisher: NeurIPS

The Connection Between Approximation, Depth Separation and Learnability in Neural Networks

Author(s): Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Published in: 2021
Publisher: COLT 2021

The Effects of Mild Over-parameterization on the Optimization Landscapeof Shallow ReLU Neural Networks

Author(s): Itay Safran, Gilad Yehudai and Ohad Shamir
Published in: 2021
Publisher: COLT 2021

Implicit Regularization in ReLU Networks with the Square Loss

Author(s): Gal Vardi and Ohad Shamir
Published in: 2021
Publisher: COLT 2021

Oracle Complexity in Nonsmooth Nonconvex Optimization

Author(s): Guy Kornowski and Ohad Shamir
Published in: 2021
Publisher: NeurIPS

Depth Separation in Norm-Bounded Infinite-Width Neural Networks

Author(s): S Parkinson, G Ongie, R Willett, O Shamir, N Srebro
Published in: 2024
Publisher: Proceedings of COLT 2024

The Implicit Bias of Benign Overfitting

Author(s): Ohad Shamir
Published in: 2022
Publisher: COLT

Are ResNets Provably Better than Linear Predictors?

Author(s): Ohad Shamir
Published in: 2018
Publisher: NeurIPS conference

On the Power and Limitations of Random Features for Understanding Neural Networks

Author(s): Gilad Yehudai and Ohad Shamir
Published in: 2019
Publisher: NeurIPS conference

Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks

Author(s): Ohad Shamir
Published in: 2019
Publisher: COLT conference

Depth Separations in Neural Networks: What is Actually Being Separated?

Author(s): Itay Safran, Ronen Eldan, Ohad Shamir
Published in: 2019
Publisher: COLT conference

The Complexity of Making the Gradient Small in Stochastic Convex Optimization

Author(s): Dylan Foster, Ayush Sekhari, Ohad Shamir, Nathan Srebro, Karthik Sridharan, Blake Woodworth
Published in: 2019
Publisher: COLT conference

Learning a Single Neuron with Gradient Methods

Author(s): Gilad Yehudai and Ohad Shamir
Published in: 2020
Publisher: COLT 2020

How Good is SGD with Random Shuffling?

Author(s): Itay Safran and Ohad Shamir
Published in: 2020
Publisher: COLT 2020

Proving the Lottery Ticket Hypothesis: Pruning is All You Need

Author(s): Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Published in: 2020
Publisher: ICML 2020

From tempered to benign overfitting in relu neural networks

Author(s): Guy Kornowski, Gilad Yehudai, Ohad Shamir
Published in: 2023
Publisher: Advances in Neural Information Processing Systems

Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems

Author(s): Itay Safran and Ohad Shamir
Published in: 2021
Publisher: NeurIPS

On the Optimal Memorization Power of ReLU Neural Networks

Author(s): Gal Vardi, Gilad Yehudai, Ohad Shamir
Published in: The Optimal Memorization Power of ReLU Neural Networks ICLR 2022, 2022
Publisher: ICLR

Size and Depth Separation in Approximating Natural Functions with Neural Networks

Author(s): Gal Vardi, Daniel Reichman, Toniann Pitassi, Ohad Shamir
Published in: 2021
Publisher: COLT 2021

Generalization in kernel regression under realistic assumptions

Author(s): Daniel Barzilai, Ohad Shamir
Published in: 2024
Publisher: Proceedings of ICML 2024

Gradient Methods Never Overfit On Separable Data

Author(s): Ohad Shamir
Published in: Journal of Machine Learning Research, 2020, ISSN 1533-7928
Publisher: None (independent electronic journal)

An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization

Author(s): G Kornowski, O Shamir
Published in: Journal of Machine Learning Research, 2024, ISSN 1533-7928
Publisher: Journal of Machine Learning Research

Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Data Manifolds

Author(s): Odelia Melamed, Gilad Yehudai, Gal Vardi
Published in: 2023
Publisher: arXiv

Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?

Author(s): Ohad Shamir
Published in: 2020
Publisher: arXiv preprint

On the Complexity of Finding Small Subgradients in Nonsmooth Optimization

Author(s): Guy Kornowski and Ohad Shamir
Published in: 2022
Publisher: arXiv

Publications

Share this page

Download