Incrementally learning new classes with generative classification

Informations projet

GenClassifier4CL

N° de convention de subvention: 101067759

DOI

10.3030/101067759

Projet clôturé

Date de signature de la CE 23 Juin 2022

Date de début 1 Octobre 2022

Date de fin 30 Septembre 2024

Financé au titre de

• Marie Skłodowska-Curie Actions (MSCA)

Coût total

Aucune donnée

Contribution de l’UE

€ 191 760,00

Coordonné par

KATHOLIEKE UNIVERSITEIT LEUVEN
Belgium

Objectif

"Learning continually from non-stationary streams of data is a key feature of natural intelligence, but an unsolved problem in deep learning. Particularly challenging for deep neural networks is the problem of ""class-incremental learning"", whereby a network must learn to distinguish classes that are not observed together. In deep learning, the default approach to classification is learning discriminative classifiers. This works great in the i.i.d. setting when all classes are available simultaneously, but when new classes must be learned incrementally, successful training of discriminative classifiers depends on workarounds such as storing data or generative replay. In a radical shift of gears, here I propose to instead address class-incremental learning with generative classification. Key advantage is that generative classifiers – unlike discriminative classifiers – do not compare classes during training, but only during inference (i.e. when making a classification decision). As a proof-of-concept, in preliminary work I showed that a naïve implementation of a generative classifier, with a separate variational autoencoder model per class and likelihood estimation through importance sampling, outperforms comparable generative replay methods. To improve the efficiency, scalability, and performance of this generative classifier, I propose four further modifications: (1) move the generative modelling objective from the raw inputs to an intermediate network layer; (2) share the encoder network between classes, but not necessarily the decoder networks; (3) use fewer importance samples for unlikely classes; and (4) make classification decisions hierarchical. This way, during my MSCA fellowship hosted in the group of Prof Tinne Tuytelaars, I hope to develop generative classification into a practical, efficient, and scalable state-of-the-art deep learning method for class-incremental learning."

Champ scientifique (EuroSciVoc)

• sciences naturelles › informatique et science de l'information › intelligence artificielle › apprentissage automatique › apprentissage profond

Mots‑clés

• Continual learning; deep neural networks; catastrophic forgetting; generative classification; class-incremental learning

Programme(s)

• HORIZON.1.2 - Marie Skłodowska-Curie Actions (MSCA) PROGRAMME PRINCIPAL
  

Thème(s)

• HORIZON-MSCA-2021-PF-01-01 - MSCA Postdoctoral Fellowships 2021
  

Régime de financement

HORIZON-TMA-MSCA-PF-EF - HORIZON TMA MSCA Postdoctoral Fellowships - European Fellowships

Appel à propositions

HORIZON-MSCA-2021-PF-01

Coordinateur

KATHOLIEKE UNIVERSITEIT LEUVEN

• Contribution nette de l'UE: € 191 760,00
• Adresse: OUDE MARKT 13 ; 3000 LEUVEN ; Belgique
• Région: Vlaams Gewest › Prov. Vlaams-Brabant › Arr. Leuven
• Type d’activité: Higher or Secondary Education Establishments
• Coût total: Aucune donnée