Multimodal Agents Grounded via Interactive Communication

Información del proyecto

MAGIC

Identificador del acuerdo de subvención: 790369

DOI

10.3030/790369

Proyecto finalizado el 30 Abril 2020

Fecha de la firma de la CE 19 Abril 2018

Fecha de inicio 1 Noviembre 2018

Fecha de finalización 31 Octubre 2020

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 158 121,60

Aportación de la UE

€ 158 121,60

Coordinado por

UNIVERSIDAD POMPEU FABRA
Spain

Objetivo

One of the main goals of artificial intelligence is to build artificial agents that can interact with humans using natural language. To fully master language, an agent needs to know how to use it to accomplish a goal; to interact with another speaker; and to refer to objects in the external reality. My research project aims at equipping an artificial agent with all these skills in one single learning framework.

Communication helps humans accomplish things in the world and cooperate with each other, resulting in continuous and incremental updating of the speakers’ knowledge state. However, traditional machine learning methods used to model language are based on static and passive regimes, and are typically not grounded in external reality. I propose a radically different research programme, based on recent advancements in training neural networks using reinforcement learning, that will enable the move from a static, fully supervised to a dynamic, interactive learning where the agents need to use language to accomplish a task in the visual world. This will dramatically accelerate the development of machines that can talk with humans.

Even though I am an established researcher in computational linguistics, with substantial contributions to the integration of language and vision, I still need to fully develop my own line of research to become a leading, independent researcher in Europe. Carrying out the present proposal at Universitat Pompeu Fabra and Facebook Artificial Intelligence Research will be a fundamental step towards achieving my goal, since my hosts are leaders in computational linguistics, machine learning, and artificial intelligence in general, and specifically in the methods needed for the present proposal. Conversely, my unique profile, bridging computational linguistics and computer vision with machine learning methods, will widen the scope and outreach of the research conducted at both groups.

Ámbito científico (EuroSciVoc)

• humanidades › lenguas y literatura › lingüística
• ciencias naturales › informática y ciencias de la información › inteligencia artificial › visión artificial
• ciencias naturales › informática y ciencias de la información › inteligencia artificial › aprendizaje automático › aprendizaje por refuerzo
• ciencias naturales › informática y ciencias de la información › inteligencia artificial › inteligencia computacional

Palabras clave

• Computational Dialogue Modelling
• End-to-end Dialogue Systems
• Multi Agent
• Language Grounding
• Language and Vision
• Interactive Learning
• Reinforcement Learning
• Language Generation
• Deep Neural Networks

Programa(s)

• H2020-EU.1.3. - EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions PROGRAMA PRINCIPAL
  
• H2020-EU.1.3.2. - Nurturing excellence by means of cross-border and cross-sector mobility
  

Tema(s)

• MSCA-IF-2017 - Individual Fellowships
  

Régimen de financiación

MSCA-IF - Marie Skłodowska-Curie Individual Fellowships (IF)

Convocatoria de propuestas

H2020-MSCA-IF-2017

Coordinador

UNIVERSIDAD POMPEU FABRA

• Aportación neta de la UEn: € 158 121,60
• Dirección: PLACA DE LA MERCE, 10-12 ; 08002 Barcelona ; España
• Región: Este › Cataluña › Barcelona
• Tipo de actividad: Higher or Secondary Education Establishments
• Coste total: € 158 121,60