In recent years, Machine Learning (ML) has made incredible strides. We now see learning algorithms applied across a wide range of fields, from engineering feats like self-driving cars to social applications that involve private data.
However, these advancements come with significant challenges:
1. Many recent breakthroughs reveal unexpected behaviors that are not well understood and sometimes contradict established knowledge. A major reason for this is that traditional ML theory takes a worst-case approach, which can be overly pessimistic. In reality, data is rarely the worst case, and experiments often show that much less data is needed than traditional theory predicts.
2. As ML applications increasingly handle private and sensitive data, it’s crucial to develop algorithms that protect this information responsibly. Although the field of Differential Privacy (DP) addresses this need, we still don’t fully understand the cost of privacy: How much more data is needed when privacy is a requirement compared to when it’s not?
Motivated by these challenges, our key question is:
**How much data is needed for learning?**
To answer this, we aim to develop a new theory of generalization that better reflects real-world learning tasks, complementing traditional approaches. We plan to build this theory around perspectives that depend on the data, the distribution, and the algorithm itself, rather than focusing solely on the worst-case scenarios of classical theory. This approach allows us to take advantage of the specific characteristics of each learning task.
We will use this new framework to explore different learning scenarios, including supervised, semi-supervised, interactive, and private learning. We believe that this work will improve the efficiency, reliability, and real-world relevance of ML. Additionally, since our research draws on ideas from various areas within computer science and mathematics, we expect it to have broader impacts beyond our field.
