Recurrent Neural Networks and Related Machines That Learn Algorithms

Project Information

AlgoRNN

Grant agreement ID: 742870

DOI

10.3030/742870

Project closed

EC signature date 12 June 2017

Start date 1 October 2017

End date 31 March 2024

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 2 500 000,00

EU contribution

€ 2 500 000,00

Coordinated by

UNIVERSITA DELLA SVIZZERA ITALIANA
Switzerland

Periodic Reporting for period 4 - AlgoRNN (Recurrent Neural Networks and Related Machines That Learn Algorithms)

Reporting period: 2022-04-01 to 2024-03-31

Summary of the context and overall objectives of the project

Our artificial recurrent neural networks (RNNs) permeate the modern world on billions of devices such as smartphones. They are used for speech recognition, translation, robotics, healthcare, etc. However, in lots of real-world tasks, RNNs do not yet live up to their full potential. Although universal in theory, in practice they fail to learn important types of algorithms. This ERC project will go beyond what's possible with today's best RNNs, by creating general practical program learners through novel RNN-like systems that address some of the biggest open RNN problems and hottest RNN research topics: (1) How can RNNs learn to control (through internal spotlights of attention) separate large short-memory structures such as sub-networks with fast weights, to improve performance on many natural short-term memory-intensive tasks which are currently hard to learn by RNNs, such as answering detailed questions on recently observed videos? (2) How can such RNN-like systems meta-learn entire learning algorithms that outperform the original learning algorithms? (3) How to achieve efficient transfer learning from one RNN-learned set of problem-solving programs to new RNN programs solving new tasks? In other words, how can one RNN-like system actively learn to exploit algorithmic information contained in the programs running on another?

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have made substantial progress on (1), (2), (3). For more details, please see the section “Project Achievements” and the separate long list of peer-reviewed publications derived from this ERC project!

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

We have achieved the goals that we set ourselves: (1) Generalize our novel Tensor-Product RNNs with fast weights [1] to self-supervision and meta-learning (learning to learn). One goal is to leverage an association mechanism to perform some level of systematic reasoning. (2) Generalize our novel method for meta-learning general reinforcement learning algorithms (MetaGenRL) [7] to support more advanced learning algorithms and replace human-engineered algorithms. Also develop stable methods to learn policies combining the information collected by many different learning agents. (3) Analyze how to make neural networks learn modular structure that supports compositionality as required for recursive function calls, and investigate what kind of inductive bias is necessary to obtain such compositional solutions. All of this is targeted towards achieving more human-like high-level reasoning.