CORDIS
EU research results

CORDIS

English EN
Deep Learning for Structured Prediction in Natural Language Processing

Deep Learning for Structured Prediction in Natural Language Processing

Objective

Deep learning is revolutionizing the field of Natural Language Processing (NLP), with breakthroughs in machine translation, speech recognition, and question answering. New language interfaces (digital assistants, messenger apps, customer service bots) are emerging as the next technologies for seamless, multilingual communication among humans and machines.

From a machine learning perspective, many problems in NLP can be characterized as structured prediction: they involve predicting structurally rich and interdependent output variables. In spite of this, current neural NLP systems ignore the structural complexity of human language, relying on simplistic and error-prone greedy search procedures. This leads to serious mistakes in machine translation, such as words being dropped or named entities mistranslated. More broadly, neural networks are missing the key structural mechanisms for solving complex real-world tasks requiring deep reasoning.

This project attacks these fundamental problems by bringing together deep learning and structured prediction, with a highly disruptive and cross-disciplinary approach. First, I will endow neural networks with a "planning mechanism" to guide structural search, letting decoders learn the optimal order by which they should operate. This makes a bridge with reinforcement learning and combinatorial optimization. Second, I will develop new ways of automatically inducing latent structure inside the network, making it more expressive, scalable and interpretable. Synergies with probabilistic inference and sparse modeling techniques will be exploited. To complement these two innovations, I will investigate new ways of incorporating weak supervision to reduce the need for labeled data.

Three highly challenging applications will serve as testbeds: machine translation, quality estimation, and dependency parsing. To maximize technological impact, a collaboration is planned with a start-up company in the crowd-sourcing translation industry.
Leaflet | Map data © OpenStreetMap contributors, Credit: EC-GISCO, © EuroGeographics for the administrative boundaries

Host institution

INSTITUTO DE TELECOMUNICACOES

Address

Campus Universitario De Santiago Universidade De Aveiro
3810 193 Gloria E Vera Cruz

Portugal

Activity type

Research Organisations

EU Contribution

€ 1 336 000

Beneficiaries (2)

Sort alphabetically

Sort by EU Contribution

Expand all

INSTITUTO DE TELECOMUNICACOES

Portugal

EU Contribution

€ 1 336 000

UNBABEL, LDA

Portugal

EU Contribution

€ 100 000

Project information

Grant agreement ID: 758969

Status

Ongoing project

  • Start date

    1 February 2018

  • End date

    31 January 2023

Funded under:

H2020-EU.1.1.

  • Overall budget:

    € 1 436 000

  • EU contribution

    € 1 436 000

Hosted by:

INSTITUTO DE TELECOMUNICACOES

Portugal