Skip to main content

A Theory for Understanding, Designing, and Training Deep Learning Systems

Publications

Neural Networks with Small Weights and Depth-Separation Barriers

Author(s): Gal Vardi and Ohad Shamir
Published in: 2020

Are ResNets Provably Better than Linear Predictors?

Author(s): Ohad Shamir
Published in: 2018

On the Power and Limitations of Random Features for Understanding Neural Networks

Author(s): Gilad Yehudai and Ohad Shamir
Published in: 2019

Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks

Author(s): Ohad Shamir
Published in: 2019

Depth Separations in Neural Networks: What is Actually Being Separated?

Author(s): Itay Safran, Ronen Eldan, Ohad Shamir
Published in: 2019

The Complexity of Making the Gradient Small in Stochastic Convex Optimization

Author(s): Dylan Foster, Ayush Sekhari, Ohad Shamir, Nathan Srebro, Karthik Sridharan, Blake Woodworth
Published in: 2019

Learning a Single Neuron with Gradient Methods

Author(s): Gilad Yehudai and Ohad Shamir
Published in: 2020

How Good is SGD with Random Shuffling?

Author(s): Itay Safran and Ohad Shamir
Published in: 2020

Proving the Lottery Ticket Hypothesis: Pruning is All You Need

Author(s): Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Published in: 2020

Gradient Methods Never Overfit On Separable Data

Author(s): Ohad Shamir
Published in: 2020

The Connection Between Approximation, Depth Separation and Learnability in Neural Networks

Author(s): Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Published in: 2021

The Effects of Mild Over-parameterization on the Optimization Landscapeof Shallow ReLU Neural Networks

Author(s): Itay Safran, Gilad Yehudai and Ohad Shamir
Published in: 2020

Implicit Regularization in ReLU Networks with the Square Loss

Author(s): Gal Vardi and Ohad Shamir
Published in: 2020

Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?

Author(s): Ohad Shamir
Published in: 2020

Size and Depth Separation in Approximating Natural Functions with Neural Networks

Author(s): Gal Vardi, Daniel Reichman, Toniann Pitassi, Ohad Shamir
Published in: 2021