Project description
Visual system inspires biologically plausible transformers for deep learning
Deep learning is a type of machine learning that teaches computers to process information the way the brain does, with multi-layer neural networks representing several levels or stages of information processing. Their increasing size and complexity has resulted in remarkable success – and increasingly prohibitive energy consumption. Biologically plausible frameworks are a promising alternative. With the support of the Marie Skłodowska-Curie Actions programme, the BiTFormer project aims to leverage the multiscale dynamics in the primary vision system to explore biologically plausible architectures for transformers, crucial elements of deep learning. Success will enable implementation in the project’s existing biologically plausible opto-analogue hardware that performs as well as digital deep learning for feedforward networks.
Objective
Deep learning (DL) has recently achieved remarkable success due to the continuous growth in model sizes. However, this growth has led to increased energy consumption. Hardware implementation of digital DL can help reduce energy usage, but the Von Neumann architecture of current DL has hindered its practical realization. In contrast, the brain exhibits energy-efficient multiscale spatiotemporal processing. Biologically plausible (BiP) frameworks have emerged as alternatives to mainstream DL. These methods use bottom-up and top-down signals, incorporating feedforward and feedback mechanisms, and local objectives instead of global error. Recently, I demonstrated that a BiP opto-analog hardware can achieve competitive performance compared to digital DL for feedforward networks. However, transformers, the backbone of current DL, are challenging to implement due to the input-dependent quadratic complexity in the transformer's attention. This project leverages the multiscale dynamics in the primary vision system to explore BiP architectures for transformers.
The project is hosted at the University of Tübingen under Matthias Bethge and Thomas Euler, who have a long-standing effort in the system identification of mouse retina via DL. The project has three objectives. First, I will extract top-down information from neural recordings of ganglion cells in the mouse retina, focusing on unique spatiotemporal features that maximally activate specific cell types. Next, I will combine top-down signals with bottom-up models of the retina using recurrent architectures with linear complexity and compare their performance in classification tasks to a vision transformer for the retina. Lastly, I propose a BiP transformer with local weight updates. I will examine the robustness of models under data distribution shifts and noise injection. A positive outcome of the project will address energy and cost issues of AI and help me progress my academic career in this interdisciplinary field.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
This project has not yet been classified with EuroSciVoc.
Be the first one to suggest relevant scientific fields and help us improve our classification service
You need to log in or register to use this function
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.2 - Marie Skłodowska-Curie Actions (MSCA)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-TMA-MSCA-PF-EF - HORIZON TMA MSCA Postdoctoral Fellowships - European Fellowships
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) HORIZON-MSCA-2023-PF-01
See all projects funded under this callCoordinator
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
72074 Tuebingen
Germany
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.