Skip to main content
Go to the home page of the European Commission (opens in new window)
English en
CORDIS - EU research results
CORDIS

Deep Learning for Holistic Inference

Project description

Bridging the gap between machine learning and human cognition

Neural networks enable machine learning algorithms to recognise underlying relationships in a data set and to perform several tasks, like image classification or speech processing. However, artificial intelligence (AI) is still far from the human ability to perform a great range of functions, like to rapidly organise a textual or visual input stream into a set of entities or to understand complex relations. To bridge that gap, the EU-funded HOLI project aims to create a new general methodology for semantic interpretation of input streams empowering deep learning algorithms with holistic inference. It will do that in a holistic approach, based on deep learning architectures, suitably developed theory and algorithms. The project will trigger enormous progress in AI technologies, enabling applications like machine reading or text-to-image analysis.

Objective

Machine learning has rapidly evolved in the last decade, significantly improving accuracy on tasks such as image classification. Much of this success can be attributed to the re-emergence of neural nets. However, learning algorithms are still far from achieving the capabilities of human cognition. In particular, humans can rapidly organize an input stream (e.g. textual or visual) into a set of entities, and understand the complex relations between those. In this project I aim to create a general methodology for semantic interpretation of input streams. Such problems fall under the structured-prediction framework, to which I have made numerous contributions. The proposal identifies and addresses three key components required for a comprehensive and empirically effective approach to the problem.
First, we consider the holistic nature of semantic interpretations, where a top-down process chooses a coherent interpretation among the vast number of options. We argue that deep-learning architectures are ideally suited for modeling such coherence scores, and propose to develop the corresponding theory and algorithms. Second, we address the complexity of the semantic representation, where a stream is mapped into a variable number of entities, each having multiple attributes and relations to other entities. We characterize the properties a model should satisfy in order to produce such interpretations, and propose novel models that achieve this. Third, we develop a theory for understanding when such models can be learned efficiently, and how well they can generalize. To achieve this, we address key questions of non-convex optimization, inductive bias and generalization. We expect these contributions to have a dramatic impact on AI systems, from machine reading of text to image analysis. More broadly, they will help bridge the gap between machine learning as an engineering field, and the study of human cognition.

Fields of science (EuroSciVoc)

Keywords

Programme(s)

Topic(s)

Funding Scheme

ERC-COG - Consolidator Grant

See all projects funded under this funding scheme

Call for proposal

(opens in new window) ERC-2018-COG

See all projects funded under this call

Host institution

TEL AVIV UNIVERSITY
Net EU contribution
€ 1 932 500,00
Total cost
€ 1 932 500,00

Beneficiaries (1)