A Theory for Understanding, Designing, and Training Deep Learning Systems

Informazioni relative al progetto

THUNDEEP

ID dell’accordo di sovvenzione: 754705

DOI

10.3030/754705

Progetto chiuso

Data della firma CE 5 Settembre 2017

Data di avvio 1 Settembre 2018

Data di completamento 31 Luglio 2025

Finanziato da

• EXCELLENT SCIENCE - European Research Council (ERC)

Costo totale

€ 1 442 360,00

Contributo UE

€ 1 442 360,00

Coordinato da

WEIZMANN INSTITUTE OF SCIENCE
Israel

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Pubblicazioni

Conference proceedings (37)

Neural Networks with Small Weights and Depth-Separation Barriers

Autori: Gal Vardi and Ohad Shamir
Pubblicato in: 2020
Editore: NeurIPS 2020

On Margin Maximization in Linear and ReLU Networks

Autori: Gal Vardi, Gilad Yehudai, Ohad Shamir
Pubblicato in: 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Numero 36, 2022
Editore: NeurIPS

Initialization-dependent sample complexity of linear predictors and neural networks

Autori: R Magen, O Shamir
Pubblicato in: 2023
Editore: Advances in Neural Information Processing Systems

When Models Don't Collapse: On the Consistency of Iterative MLE

Autori: D Barzilai, O Shamir
Pubblicato in: 2025
Editore: NeurIPS 2025

Implicit Regularization Towards Rank Minimization in ReLU Networks

Autori: Nadav Timor, Gal Vardi and Ohad Shamir
Pubblicato in: 2023
Editore: ALT

The Sample Complexity of One-Hidden-Layer Neural Networks

Autori: Gal Vardi, Ohad Shamir and Nathan Srebro
Pubblicato in: 2022
Editore: NeurIPS

Learning a Single Neuron with Bias Using Gradient Descent

Autori: Gal Vardi, Gilad Yehudai, Ohad Shamir
Pubblicato in: 2021
Editore: NeurIP

Deterministic nonsmooth nonconvex optimization

Autori: M Jordan, G Kornowski, T Lin, O Shamir, M Zampetakis
Pubblicato in: 2023
Editore: Proceedings of COLT 2023

Width is Less Important than Depth in ReLU Neural Networks

Autori: Gal Vardi, Gilad Yehudai and Ohad Shamir
Pubblicato in: 2022
Editore: COLT

Open Problem: Anytime Convergence Rate of Gradient Descent

Autori: G Kornowski, O Shamir
Pubblicato in: 2024
Editore: Proceedings of COLT 2024

Gradient Methods Provably Converge to Non-Robust Networks

Autori: Gal Vardi, Gilad Yehudai and Ohad Shamir
Pubblicato in: 2022
Editore: NeurIPS

Reconstructing Training Data from Trained Neural Networks

Autori: Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, Michal Irani
Pubblicato in: 2022
Editore: NeurIPS

The Connection Between Approximation, Depth Separation and Learnability in Neural Networks

Autori: Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Pubblicato in: 2021
Editore: COLT 2021

The Effects of Mild Over-parameterization on the Optimization Landscapeof Shallow ReLU Neural Networks

Autori: Itay Safran, Gilad Yehudai and Ohad Shamir
Pubblicato in: 2021
Editore: COLT 2021

Implicit Regularization in ReLU Networks with the Square Loss

Autori: Gal Vardi and Ohad Shamir
Pubblicato in: 2021
Editore: COLT 2021

Oracle Complexity in Nonsmooth Nonconvex Optimization

Autori: Guy Kornowski and Ohad Shamir
Pubblicato in: 2021
Editore: NeurIPS

Depth Separation in Norm-Bounded Infinite-Width Neural Networks

Autori: S Parkinson, G Ongie, R Willett, O Shamir, N Srebro
Pubblicato in: 2024
Editore: Proceedings of COLT 2024

The Implicit Bias of Benign Overfitting

Autori: Ohad Shamir
Pubblicato in: 2022
Editore: COLT

Reconstructing Training Data from Multiclass Neural Networks

Autori: Buzaglo, Gon; Haim, Niv; Yehudai, Gilad; Vardi, Gal; Irani, Michal
Pubblicato in: Workshop on the pitfalls of limited data and computation for Trustworthy ML, ICLR 2023, 2023
Editore: ICLR

Logarithmic Width Suffices for Robust Memorization

Autori: A Egosi, G Yehudai, O Shamir
Pubblicato in: 2025
Editore: COLT 2025

Are ResNets Provably Better than Linear Predictors?

Autori: Ohad Shamir
Pubblicato in: 2018
Editore: NeurIPS conference

On the Power and Limitations of Random Features for Understanding Neural Networks

Autori: Gilad Yehudai and Ohad Shamir
Pubblicato in: 2019
Editore: NeurIPS conference

Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks

Autori: Ohad Shamir
Pubblicato in: 2019
Editore: COLT conference

Depth Separations in Neural Networks: What is Actually Being Separated?

Autori: Itay Safran, Ronen Eldan, Ohad Shamir
Pubblicato in: 2019
Editore: COLT conference

The Complexity of Making the Gradient Small in Stochastic Convex Optimization

Autori: Dylan Foster, Ayush Sekhari, Ohad Shamir, Nathan Srebro, Karthik Sridharan, Blake Woodworth
Pubblicato in: 2019
Editore: COLT conference

Learning a Single Neuron with Gradient Methods

Autori: Gilad Yehudai and Ohad Shamir
Pubblicato in: 2020
Editore: COLT 2020

How Good is SGD with Random Shuffling?

Autori: Itay Safran and Ohad Shamir
Pubblicato in: 2020
Editore: COLT 2020

Proving the Lottery Ticket Hypothesis: Pruning is All You Need

Autori: Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, Ohad Shamir
Pubblicato in: 2020
Editore: ICML 2020

From tempered to benign overfitting in relu neural networks

Autori: Guy Kornowski, Gilad Yehudai, Ohad Shamir
Pubblicato in: 2023
Editore: Advances in Neural Information Processing Systems

Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems

Autori: Itay Safran and Ohad Shamir
Pubblicato in: 2021
Editore: NeurIPS

On the Optimal Memorization Power of ReLU Neural Networks

Autori: Gal Vardi, Gilad Yehudai, Ohad Shamir
Pubblicato in: The Optimal Memorization Power of ReLU Neural Networks ICLR 2022, 2022
Editore: ICLR

Size and Depth Separation in Approximating Natural Functions with Neural Networks

Autori: Gal Vardi, Daniel Reichman, Toniann Pitassi, Ohad Shamir
Pubblicato in: 2021
Editore: COLT 2021

Generalization in kernel regression under realistic assumptions

Autori: Daniel Barzilai, Ohad Shamir
Pubblicato in: 2024
Editore: Proceedings of ICML 2024

Beyond Benign Overfitting in Nadaraya-Watson Interpolators

Autori: D Barzilai, G Kornowski, O Shamir
Pubblicato in: 2025
Editore: NeurIPS 2025

Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses

Autori: Buzaglo, Gon; Haim, Niv; Yehudai, Gilad; Vardi, Gal; Oz, Yakir; Nikankin, Yaniv; Irani, Michal
Pubblicato in: 37th Conference on Neural Information Processing Systems (NeurIPS 2023), 2023
Editore: Neural Information Processing Systems

The oracle complexity of simplex-based matrix games: Linear separability and nash equilibria

Autori: G Kornowski, O Shamir
Pubblicato in: 2025
Editore: COLT 2025

On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization

Autori: Guy Kornowski, Swati Padmanabhan, Ohad Shamir
Pubblicato in: 2024
Editore: NeurIPS Workshop on Optimization for Machine Learning 2024

Peer reviewed articles (2)

Gradient Methods Never Overfit On Separable Data

Autori: Ohad Shamir
Pubblicato in: Journal of Machine Learning Research, 2020, ISSN 1533-7928
Editore: None (independent electronic journal)

An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization

Autori: G Kornowski, O Shamir
Pubblicato in: Journal of Machine Learning Research, 2024, ISSN 1533-7928
Editore: Journal of Machine Learning Research

Other (5)

Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Data Manifolds

Autori: Odelia Melamed, Gilad Yehudai, Gal Vardi
Pubblicato in: 2023
Editore: arXiv

Simple Relative Deviation Bounds for Covariance and Gram Matrices

Autori: Daniel Barzilai, Ohad Shamir
Pubblicato in: 2024
Editore: arXiv

Hardness of learning fixed parities with neural networks

Autori: Itamar Shoshani, Ohad Shamir
Pubblicato in: 2025
Editore: arXiv

Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?

Autori: Ohad Shamir
Pubblicato in: 2020
Editore: arXiv preprint

On the Complexity of Finding Small Subgradients in Nonsmooth Optimization

Autori: Guy Kornowski and Ohad Shamir
Pubblicato in: 2022
Editore: arXiv