Optimizing for Generalization in Machine Learning

Project Information

OPTGEN

Grant agreement ID: 101078075

DOI

10.3030/101078075

EC signature date 7 December 2022

Start date 1 October 2023

End date 30 September 2028

Funded under

• European Research Council (ERC)

Total cost

€ 1 494 375,00

EU contribution

€ 1 494 375,00

Coordinated by

TEL AVIV UNIVERSITY
Israel

Periodic Reporting for period 1 - OPTGEN (Optimizing for Generalization in Machine Learning)

Reporting period: 2023-10-01 to 2026-03-31

Summary of the context and overall objectives of the project

Recent advances in the field of machine learning (ML) and artificial intelligence (AI) are revolutionizing an ever-growing variety of domains, ranging from statistical learning algorithms in computer vision and natural language processing all the way to reinforcement learning algorithms in autonomous driving and conversational AI. However, many of these breakthroughs demonstrate phenomena that lack explanations, and sometimes even contradict conventional wisdom. Perhaps the greatest mystery of modern ML—and arguably, one of the greatest mysteries of all of modern computer science—is the question of generalization: why do these immensely complex prediction rules successfully apply to future unseen instances? Apart from the pure scientific curiosity it stimulates, this lack of understanding poses a significant obstacle to widening the applicability of ML to critical applications, like in healthcare or autonomous driving, where the cost of error is disastrous.

The broad goal of this project is to tackle the generalization mystery in the context of both statistical learning and reinforcement learning, focusing on optimization algorithms being the de facto contemporary standard in training learning models. Our methodology points out to inherent shortcomings of widely accepted viewpoints with regard to generalization of optimization-based learning algorithms, and takes a crucially different approach that targets the optimization algorithm itself; building bottom-up from fundamental and tractable optimization models, we identify intrinsic properties and develop algorithmic methodologies that enable optimization to effectively generalize in modern statistical, sequential decision-making, and reinforcement-learning scenarios.

A successful outcome would not only lead to a timely and crucial shift in the way the research community approaches generalization of contemporary optimization-based ML, but it may also significantly transform the way we develop practical, efficient and reliable learning systems.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

At the midpoint of the project, we have made substantial progress toward understanding one of the most fundamental questions in modern machine learning and AI: why do learning algorithms that are trained on limited data perform so well on new, unseen examples? Our research explores this question through the lens of optimization: the mathematical process by which machine learning models are trained, and has led to several important and sometimes surprising discoveries.

A central achievement is a new theoretical understanding of how stochastic (randomized) training methods generalize. We discovered that stochastic gradient descent, the algorithmic paradigm that drives modern machine learning, performs well not because it finds the simplest model or explicitly avoids overfitting, but because of intrinsic algorithmic properties that cause it to generalize reliably even in very high-dimensional settings. This insight reshapes long-held assumptions about how learning works and represents a major advance in the theoretical understanding of generalization. We also showed that while a single pass of stochastic training achieves robust performance, repeating passes over the same data can quickly lead to overfitting, revealing a delicate balance between optimization and generalization even in fundamental, extensively studied learning scenarios.

We have also made notable progress in understanding learning under limited feedback and uncertainty, including in reinforcement learning (RL), where an agent learns through interaction with its environment. Our work developed efficient algorithms that generalize across dynamic settings, even when the model of the environment is inaccurate or misspecified, advancing the theoretical understanding of agnostic RL as a robust framework for learning from imperfect models. In parallel, we achieved breakthrough results in multiclass classification with limited feedback, proving for the first time that optimal, fast learning rates are achievable even in the so-called “bandit setting,” where only partial outcome information is revealed. These advances resolve long-standing open questions and provide a rigorous foundation for learning and decision-making algorithms that generalize reliably under uncertainty.

In terms of methodologies and algorithms for reliable generalization, we advanced the study of parameter-free and tuning-robust optimization: methods that can automatically adapt to the learning problem without requiring manual adjustment of hyperparameters such as the learning rate. These algorithms simplify the practical use of machine learning systems while maintaining optimal generalization, making them both efficient and reliable in real-world applications. Taken together, these achievements mark a significant step forward in understanding how optimization drives generalization and in developing more adaptive, principled, and robust approaches to generalization in modern machine learning and AI.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The project has made significant progress beyond the state of the art in our theoretical understanding of how optimization algorithms generalize, both in supervised and reinforcement learning settings. We resolved several long-standing open questions, including why learning algorithms perform remarkably well out-of-sample despite their simplicity and lack of explicit capacity control, and under what circumstances additional training leads to overfitting rather than improvement. These findings reshape prevailing views on the foundations of learning theory and are already influencing how researchers think about model training, robustness, and the interpretation of overparameterization.

Our work has also introduced new concrete methodologies for reliable generalization at large scale, including advances in adaptive, parameter-free, and tuning-robust optimization. These results provide a concrete bridge between theory and practice, offering learning methods that are easier to deploy and less sensitive to misspecification of hyperparameters, being an important step toward more reliable and accessible AI technologies.

The broader societal impact of these advances lies in their potential to make machine learning systems more transparent, predictable, and trustworthy. By clarifying the principles that govern generalization and stability, the project contributes to the development of AI systems that behave more reliably across diverse and evolving environments. In the longer term, these insights will support the creation of learning technologies that require less supervision and are less prone to failure: these qualities are essential for the responsible and safe integration of AI into high-stakes domains such as healthcare, education, and scientific discovery.