Skip to main content

Practically Relevant Theory of Deep Learning

Objective

One of the most significant recent developments in applied machine learning has been the resurgence of ``deep learning'', usually in the form of artificial neural networks. The empirical success of deep learning is stunning, and deep learning based systems have already led to breakthroughs in computer vision and speech recognition. In contrast, from the theoretical point of view, by and large, we do not understand why deep learning is at all possible, since most state of
the art theoretical results show that deep learning is computationally hard.

Bridging this gap is a great challenge since it involves proficiency in several theoretic fields (algorithms, complexity, and statistics) and at the same time requires a good understanding of real world practical problems and the ability to conduct applied research. We believe that a good theory must lead to better practical algorithms. It should also broaden the applicability of learning in general, and deep learning in particular, to new domains. Such a practically relevant theory may also lead to a fundamental paradigm shift in the way we currently analyze the complexity of algorithms.

Previous works by the PI and his colleagues and students have provided novel ways to analyze the computational complexity of learning algorithms and understand the tradeoffs between data and computational time. In this proposal, in order to bridge the gap between theory and practice, I suggest a departure from worst-case analyses and the development of a more optimistic, data dependent, theory with ``grey'' components. Success will lead to a breakthrough in our understanding of learning at large with significant potential for impact on the field of machine learning and its applications.

Field of science

  • /natural sciences/computer and information sciences/artificial intelligence/machine learning/deep learning

Call for proposal

ERC-2015-STG
See other projects for this call

Funding Scheme

ERC-STG - Starting Grant

Host institution

THE HEBREW UNIVERSITY OF JERUSALEM
Address
Edmond J Safra Campus Givat Ram
91904 Jerusalem
Israel
Activity type
Higher or Secondary Education Establishments
EU contribution
€ 1 342 500

Beneficiaries (1)

THE HEBREW UNIVERSITY OF JERUSALEM
Israel
EU contribution
€ 1 342 500
Address
Edmond J Safra Campus Givat Ram
91904 Jerusalem
Activity type
Higher or Secondary Education Establishments