Theoretical Understanding of Classic Learning Algorithms

Descripción del proyecto

Desarrollar algoritmos de aprendizaje clásicos más rápidos

El avance de la tecnología de aprendizaje automático ha aportado beneficios inigualables a diversos sectores, facilitando así una automatización mejorada y más accesible. Actualmente, los algoritmos de aprendizaje se clasifican en aprendizaje profundo, que destaca en entornos con abundancia de datos, y algoritmos clásicos, más adecuados para entornos con escasez de datos. En este contexto, el equipo del proyecto TUCLA, financiado por el Consejo Europeo de Investigación, pretende profundizar en los algoritmos clásicos y mejorar su velocidad y eficacia. El equipo del proyecto se centra en el estudio de algoritmos fundamentales como el «bagging» y el «boosting» para establecer un marco teórico de aprendizaje que permita diseñar algoritmos innovadores de «boosting» y «bagging».

Objetivo

Machine learning has evolved from being a relatively isolated discipline to have a disruptive influence on all areas of science, industry and society. Learning algorithms are typically classified into either deep learning or classic learning, where deep learning excels when data and computing resources are abundant, whereas classic algorithms shine when data is scarce. In the TUCLA project, we expand our theoretical understanding of classic machine learning, with a particular emphasis on two of the most important such algorithms, namely Bagging and Boosting. As a result of this study, we shall provide faster learning algorithms that require less training data to make accurate predictions. The project accomplishes this by pursuing several objectives:

1. We will establish a novel learning theoretic framework for proving generalization bounds for learning algorithms. Using the framework, we will design new Boosting algorithms and prove that they make accurate predictions using less training data than what was previously possible. Moreover, we complement these algorithms by generalization lower bounds, proving that no other algorithm can make better use of data.

2. We will design parallel versions of Boosting algorithms, thereby allowing them to be used in combination with more computationally expensive base learning algorithms. We conjecture that success in this direction may lead to Boosting playing a more central role also in deep learning.

3. We will explore applications of the classic Bagging heuristic. Until recently, Bagging was not known to have significant theoretical benefits. However, recent pioneering work by the PI shows that Bagging is an optimal learning algorithm in an important learning setup. Using these recent insights, we will explore theoretical applications of Bagging in other important settings.