Project description
Understanding residual networks in deep learning theory
Residual neural networks have excelled in AI for the past decade. However, we understand little about the mechanisms behind feature learning in these networks. Supported by the Marie Skłodowska-Curie Actions programme, the SM-DeepResNet project will investigate residual learning by applying statistical mechanics to deep, fully connected architectures. It will create a framework for studying deep residual networks during feature learning, providing predictive models to explain their behaviour. The project will also tackle key challenges in deep learning theory, including calculating exponents of the neural scaling law and implementing hyperparameter transferability. Ultimately, it will clarify how structured data, network width and depth impact neural networks, aiding the development of next-generation AI architectures.
Objective
"Since their introduction a decade ago, neural networks with residual connections have come to dominate the field of artificial neural architectures thanks to their outstanding performance, and are nowadays the building blocks of most state-of-the-art, scalable AI systems. Nevertheless, despite their widespread use, relatively little is known about the quantitative mechanisms underlying feature learning in these architectures and how to improve their inference capabilities.
To this aim, ""Statistical Mechanics of Deep Residual Networks in the Feature Learning Regime"" (SM-DeepResNet) lays the groundwork to close this gap and achieve a broad theoretical understanding of residual learning. Leveraging recent successful applications of statistical mechanics to the study of deep fully connected architectures, SM-DeepResNet will contributes to: (i) building a novel statistical-mechanics-inspired framework to study deep residual nets in the feature learning regime, providing low-dimensional predictive models that theoretically describe their behavior through scaling limits; (ii) addressing two major open problems in deep learning theory: the computation of the Neural Scaling Law exponents and the implementation of hyperparameter transferability, by analytically exploiting the predictive capabilities of the aforementioned foundational theories.
SM-DeepResNet synergizes the applicant’s core competency in computational physics with the supervisors' expertise in statistical mechanics (host institution) and in deep learning mathematics (associated partner).
The main purpose of this broad program is to unveil the entangled roles of structured data, width and depth in overparameterized deep residual neural networks, ultimately contributing to the development of effective strategies for the next generation of AI architectures."
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
You need to log in or register to use this function
Keywords
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Project’s keywords as indicated by the project coordinator. Not to be confused with the EuroSciVoc taxonomy (Fields of science)
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.2 - Marie Skłodowska-Curie Actions (MSCA)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-TMA-MSCA-PF-GF - HORIZON TMA MSCA Postdoctoral Fellowships - Global Fellowships
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) HORIZON-MSCA-2025-PF
See all projects funded under this callCoordinator
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
43121 PARMA
Italy
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.