Final Report Summary - SUBLRN (Information-optimal machine learning)
This final report pertains to project ''Information-optimal machine learning" carried out at the Technion, PI: Elad Hazan, and generously funded through the European Research Council. The project was discontinued after 2 years of activity, due to relocation of the PI to the United States, Princeton University.
The main question tackled in this project is: "can we solve optimization problems in using computational resources proportional to the information necessary to represent and verify the solution?".
The interplay between information and computation is at the core of the hardest problems of computer science and mathematics.
However, statistical problems arising in machine learning exhibit a much more attainable version of this question, which has been a major focus of this research project. In the past few years we have been able to successfully design algorithms that run in time proportional to the information theoretic limit of verifying a solution. These algorithms run in time which is lesser than time to perform even a linear scan of the input, thus are called sublinear optimization algorithms. Surprisingly, this phenomenon is not limited to esoteric or contrived computational models, but rather hold for perhaps the most widely used optimization algorithms in use today: linear classification and kernel machines.
As part of our project, several other efficient algorithms for optimization in machine learning were developed, that are now part of standard software libraries and taught in introductory courses on optimization for machine learning.
The main question tackled in this project is: "can we solve optimization problems in using computational resources proportional to the information necessary to represent and verify the solution?".
The interplay between information and computation is at the core of the hardest problems of computer science and mathematics.
However, statistical problems arising in machine learning exhibit a much more attainable version of this question, which has been a major focus of this research project. In the past few years we have been able to successfully design algorithms that run in time proportional to the information theoretic limit of verifying a solution. These algorithms run in time which is lesser than time to perform even a linear scan of the input, thus are called sublinear optimization algorithms. Surprisingly, this phenomenon is not limited to esoteric or contrived computational models, but rather hold for perhaps the most widely used optimization algorithms in use today: linear classification and kernel machines.
As part of our project, several other efficient algorithms for optimization in machine learning were developed, that are now part of standard software libraries and taught in introductory courses on optimization for machine learning.