Works in the domain of supervised learning with statistics and computations beyond worst-case analysis:
To set goals in designing optimal learning algorithms, the first step is determining theoretically which is the maximum accuracy and reliability achievable in a learning problem. The challenge in this domain is to devise tight statistical/computational bounds and the resulting trade-offs on accuracy and reliability accounting for a wide set of ML problems, the high dimensionality/structure of the data, and more recent learning scenarios.
In the context of WP1, I worked with the Ph.D. student Gaspard Beugnot and Rémi Jézéquel and other collaborators, to analyze the statistical and computational behavior of important classes of supervised learning problems, with the goal of obtaining guarantees that are usable in practice, i.e. beyond worst-case analysis.
* "Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization" Gaspard Beugnot, Julien Mairal, Alessandro Rudi. NeurIPS 2021. (December 2021).
In this work, (a) we show how to achieve a fast statistical convergence in terms of learning rates for a wide class of supervised learning problems based on generalized self-concordant losses (b) we go beyond the worst-case rate by using a more advanced regularization technique, beyond Tikhonov regularization.
This approach could lead to a new class of learning algorithms with guarantees and reduced computational complexity.
* "Mixability made efficient: Fast online multiclass logistic regression", Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi. NeurIPS 2021. (December 2021).
In the same spirit, we devise here a new approach that allows achieving optimal rates and, at the same time, reduced computational complexity, for the problem of multiclass logistic regression, in the more difficult context of online learning, where it is not possible to do the standard assumptions of the statistical i.i.d. setting.
Works intending to extend results from supervised learning to structured prediction settings:
Instead of reinventing ad-hoc theories and algorithms for each different type of data and learning problem, the challenge here is to derive a unified framework for structured prediction problems, which is reliable and cost-effective. In particular, the approach taken here, is to provide for structured prediction problems the same reliability guarantees, generality, and cost-effective algorithms that we achieved for standard supervised learning and in particular for the results achieved in the rest of the project.
In this context, I worked with the PhD student Vivien Cabannes, and collaborators, to extend to structured prediction, existing refined results that are known for standard supervised learning techniques. We did this in a theoretical framework that is purposely flexible enough that should be able to integrate easily the future results we will obtain in the domain of supervised learning with statistics and computations beyond worst-case analysis.
* "Fast Rates for Structured Prediction" Vivien Cabannes, Francis Bach, Alessandro Rudi. Colt 2021. (Aug 2021).
This work extends to the Structured Prediction setting, the fast rates beyond the worst-case scenario, that have been derived in the case of supervised learning in the context of single output prediction (i.e. w.r.t. functions whose output is a real number).
* "Disambiguation of Weak Supervision leading to Exponential Convergence rates" Vivien Cabannes, Francis Bach, Alessandro Rudi. ICML 2021. (Jul 2021).
In this work, we study a very important subcase of structured prediction, i.e. "weak supervision" providing an analysis that goes beyond the worst case and allows to derive exponential convergence rates for suitable algorithms.
