Information Theory beyond Communications: Distributed Representations and Deep Learning

Información del proyecto

STRUDEL

Identificador del acuerdo de subvención: 797805

DOI

10.3030/797805

Proyecto cerrado

Fecha de la firma de la CE 5 Abril 2018

Fecha de inicio 1 Septiembre 2018

Fecha de finalización 31 Agosto 2020

Financiado con arreglo a

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coste total

€ 171 349,20

Aportación de la UE

€ 171 349,20

Coordinado por

CENTRALESUPELEC
France

Objetivo

Deep learning is an enormously successful recent paradigm with record-breaking performance in numerous applications. Individual autoencoders (AEs) of a multilayer neural network are trained to convert high-dimensional inputs into low-dimensional codes that allow the reconstruction of the input. Although some explanations appear to be solidly grounded, there is no mathematical understanding of the AE learning process. This project is a collaborative endeavor of researchers with strong complementary backgrounds. Its main innovation is the idea to capitalize on powerful and fertile concepts from information theory (expertise of researcher) in order to advance the state of the art in deep learning (expertise of supervisor at TC). The innovative research work is motivated by our recent insight that there is an intimate relationship between AEs, generative adversarial nets and the information bottleneck method. This method is a model-free approach for extracting information from observed variables that are relevant to hidden representations or labels and will serve as basic building block for an information theory of representation learning. The planned objectives are split into 3 workpackages: 1) information-theoretic criteria and statistical tradeoffs for extracting good representations, 2) structured architectures/algorithms for learning, 3) use of stochastic complexity to assess the descriptive power (model selection) of deep neural networks. Accomplishing the challenging goals of this proposal requires a variety of methodologies with a rich potential for transfer of knowledge between the involved fields of information theory, statistics and machine learning. Our new framework is expected to bridge the gap between theory and practice to facilitate a more thorough understanding and hence improved design of deep learning architectures. The fellow researcher is coordinating the LIA Lab of the CNRS (started in 2017) where he is collaborating with the supervisor at TC

Ámbito científico (EuroSciVoc)

• ciencias naturales › informática y ciencias de la información › inteligencia artificial › aprendizaje automático › aprendizaje profundo

Palabras clave

• machine learning
• deep learning
• information theory
• autoencoder
• unsupervised learning
• supervised learning
• learning algorithms
• distributed representations
• neural networks

Programa(s)

• H2020-EU.1.3. - EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions PROGRAMA PRINCIPAL
  
• H2020-EU.1.3.2. - Nurturing excellence by means of cross-border and cross-sector mobility
  

Tema(s)

• MSCA-IF-2017 - Individual Fellowships
  

Régimen de financiación

MSCA-IF - Marie Skłodowska-Curie Individual Fellowships (IF)

Convocatoria de propuestas

H2020-MSCA-IF-2017

Coordinador

CENTRALESUPELEC

• Aportación neta de la UEn: € 171 349,20
• Dirección: PLATEAU DE MOULON 3 RUE JOLIOT CURIE ; 91192 GIF SUR YVETTE ; Francia
• Tipo de actividad: Higher or Secondary Education Establishments
• Coste total: € 171 349,20

Socios (1)

Socio

UNIVERSITE DE MONTREAL

Canadá

• Aportación neta de la UEn: € 0,00
• Dirección: CP 6128 Station Centre Ville ; H3C3J7 Montreal
• Tipo de actividad: Higher or Secondary Education Establishments
• Coste total: € 78 811,20