Structured Physics-Inspired Representations and dAta models for efficient Learning

Description du projet

Une solution pour un apprentissage automatique efficace et durable

La croissance rapide et la diversité des avancées dans le domaine de l’IA et de l’apprentissage automatique ont mené à l’adoption généralisée des technologies de l’IA et au développement de nombreux grands modèles de langage, tels que ChatGPT. Bien que ChatGPT soit largement utilisé, il nécessite une puissance de calcul et une énergie considérables, ce qui met en évidence les défis liés à l’efficacité et à la durabilité de ces systèmes. Soutenu par le programme Actions Marie Skłodowska-Curie, le projet SPIRAL entend mettre au point des modèles d’apprentissage automatique structurés et ciblés, à la fois plus efficaces et plus faciles à interpréter. Le projet permettra de mieux comprendre comment les données structurées sont traitées et interprétées en interne, tout en optimisant les architectures actuelles pour améliorer l’efficacité et la durabilité de la formation.

Objectif

Artificial Intelligence (AI) is set to revolutionize technology and society. While the fast improvement and adoption of AI comes with tremendous potential, it also brings significant challenges. The rise of so-called ‘Large Language Models’ such as ChatGPT, which require tremendous computational resources, notably highlights the need for more efficient architectures to address sustainability and sovereignty concerns.
In this context, the SPIRAL project—Structured Physics-Inspired Representations and dAta models for efficient Learning—aims to create more interpretable, efficient, and targeted machine learning models by focusing on the role of structure. Indeed, despite advancements in Machine Learning research, the field has yet to fully understand how models process and build internal representations from structured data. SPIRAL seeks to close this gap by identifying how current architectures use structure in data and subsequently by developing better training protocols and tailored architectures, opening an alternative to the blind increase of model size and complexity.
To achieve its goals, SPIRAL will take a dual, physics-inspired approach. First, it will establish an in-silico laboratory to explore the role of structure in data by developing a tunable model of synthetic structured data and conducting targeted experiments on modern machine learning architectures. Second, it will leverage insights from the statistical physics of disordered systems to understand the emergent structure in the solution landscape of asymmetric neural networks, ultimately proposing innovative architectures that make use of this internal structure to build meaningful internal representations of data.
This interdisciplinary strategy, developed within the fast-growing Bocconi Institute for Data Science and Analytics, will provide new methods for building efficient models aligning with Europe’s goals of sustainability and technological independence.