Structured Physics-Inspired Representations and dAta models for efficient Learning

Descrizione del progetto

Soluzione per un apprendimento automatico efficiente e sostenibile

La rapida crescita e la diversità dei progressi nel campo dell’IA e dell’apprendimento automatico hanno portato a un’adozione diffusa delle tecnologie di IA e allo sviluppo di molti modelli linguistici di grandi dimensioni, come ChatGPT. ChatGPT è ampiamente utilizzato, ma richiede un’immensa potenza di calcolo ed energia, evidenziando le sfide legate all’efficienza e alla sostenibilità di questi sistemi. Sostenuto dal programma di azioni Marie Skłodowska-Curie, il progetto SPIRAL intende sviluppare modelli di apprendimento automatico strutturati e mirati, più efficienti e facili da interpretare. Il progetto approfondirà la comprensione del modo in cui i dati strutturati vengono elaborati e interpretati internamente, ottimizzando al contempo le architetture attuali per migliorare l’efficienza e la sostenibilità della formazione.

Obiettivo

Artificial Intelligence (AI) is set to revolutionize technology and society. While the fast improvement and adoption of AI comes with tremendous potential, it also brings significant challenges. The rise of so-called ‘Large Language Models’ such as ChatGPT, which require tremendous computational resources, notably highlights the need for more efficient architectures to address sustainability and sovereignty concerns.
In this context, the SPIRAL project—Structured Physics-Inspired Representations and dAta models for efficient Learning—aims to create more interpretable, efficient, and targeted machine learning models by focusing on the role of structure. Indeed, despite advancements in Machine Learning research, the field has yet to fully understand how models process and build internal representations from structured data. SPIRAL seeks to close this gap by identifying how current architectures use structure in data and subsequently by developing better training protocols and tailored architectures, opening an alternative to the blind increase of model size and complexity.
To achieve its goals, SPIRAL will take a dual, physics-inspired approach. First, it will establish an in-silico laboratory to explore the role of structure in data by developing a tunable model of synthetic structured data and conducting targeted experiments on modern machine learning architectures. Second, it will leverage insights from the statistical physics of disordered systems to understand the emergent structure in the solution landscape of asymmetric neural networks, ultimately proposing innovative architectures that make use of this internal structure to build meaningful internal representations of data.
This interdisciplinary strategy, developed within the fast-growing Bocconi Institute for Data Science and Analytics, will provide new methods for building efficient models aligning with Europe’s goals of sustainability and technological independence.