Periodic Reporting for period 1 - ALEPH (Autonomous learning agents in Physics)
Période du rapport: 2023-03-01 au 2025-02-28
It is now widely understood that machine learning (ML) holds enormous potential across almost every area of society. One particularly promising field is the application of these models to the study of Sciences, and Physics in particular. With these machines, we can create better predictions when analyzing experimental data or, for instance, perform improved simulations of physical systems, as needed in areas like weather forecasting. However, ML holds an even greater potential: the ability to discover new Physics directly from experimental data. This means that, given access to experimental observations—or even direct control over the experiment—the machine could learn to understand and describe the underlying physical processes, connecting them to our existing theories. To achieve this, we must develop models that are not only good at predicting outcomes but also capable of providing insights in a language that humans can understand.
The main objective of the project “Active Learning Agents in Physics” (ALEPH) was to develop autonomous learning agents for Physics, with a particular focus on their interpretability. This refers to designing learning machines that are not only powerful and robust in the tasks they are trained for but that can also assist in the process of scientific discovery. In ALEPH, I aimed to link scientific discovery to the interpretability of ML models. A model is considered interpretable if we can easily understand why and how a given prediction was made. Thus, if an ML model can perform accurate predictions about a physical process, it implies that important information about the process is encoded within the model. If the model is interpretable, we can access this information and use it to deepen our understanding of the physics behind the studied system.
The main objective of the project “Active Learning Agents in Physics” (ALEPH) was to develop autonomous learning agents for Physics, with a particular focus on their interpretability. This refers to designing learning machines that are not only powerful and robust in the tasks they are trained for but that can also assist in the process of scientific discovery. In ALEPH, I aimed to link scientific discovery to the interpretability of ML models. A model is considered interpretable if we can easily understand why and how a given prediction was made. Thus, if an ML model can perform accurate predictions about a physical process, it implies that important information about the process is encoded within the model. If the model is interpretable, we can access this information and use it to deepen our understanding of the physics behind the studied system.
During the ALEPH project, I published 7 papers in international journals and shared 3 pre-prints currently under review. I also presented my work at several international conferences. The work performed can be separated in three main areas:
1) Reinforcement learning in target search problems
A central theme of the project was applying reinforcement learning (RL) to target search tasks, where agents aim to capture as many targets as possible within a limited time. Our work established theoretical links between RL and search theory, and demonstrated numerically that RL can yield optimal strategies in diverse scenarios such as foraging, active matter, and search with resets. A key strength of our approach is the interpretability of the trained agents’ strategies, offering insights into the underlying problem.
2) Interpretable machine learning for anomalous diffusion.
Building on my prior expertise, I developed ML tools to autonomously extract physical parameters from experimental data, especially for stochastic processes like anomalous diffusion—common in biophysical systems. We showed how these tools must account for the randomness of the data to remain physically meaningful. I also organized the second edition of the Anomalous Diffusion (AnDi) Challenge (www.andi-challenge.org) with over 70 participants, whose results will be summarized in an upcoming publication. Additionally, I introduced a novel method to analyze diffusion properties frame-by-frame, enhancing resolution in trajectory analysis.
3) Generative models for quantum computing
Finally, I worked on applying state-of-the-art ML—particularly denoising diffusion models—to quantum circuit generation, a key bottleneck in practical quantum computing. Our results show these models can efficiently translate quantum operations into hardware-compatible gate sequences, offering a promising route to overcome current limitations and enable broader algorithmic applications.
1) Reinforcement learning in target search problems
A central theme of the project was applying reinforcement learning (RL) to target search tasks, where agents aim to capture as many targets as possible within a limited time. Our work established theoretical links between RL and search theory, and demonstrated numerically that RL can yield optimal strategies in diverse scenarios such as foraging, active matter, and search with resets. A key strength of our approach is the interpretability of the trained agents’ strategies, offering insights into the underlying problem.
2) Interpretable machine learning for anomalous diffusion.
Building on my prior expertise, I developed ML tools to autonomously extract physical parameters from experimental data, especially for stochastic processes like anomalous diffusion—common in biophysical systems. We showed how these tools must account for the randomness of the data to remain physically meaningful. I also organized the second edition of the Anomalous Diffusion (AnDi) Challenge (www.andi-challenge.org) with over 70 participants, whose results will be summarized in an upcoming publication. Additionally, I introduced a novel method to analyze diffusion properties frame-by-frame, enhancing resolution in trajectory analysis.
3) Generative models for quantum computing
Finally, I worked on applying state-of-the-art ML—particularly denoising diffusion models—to quantum circuit generation, a key bottleneck in practical quantum computing. Our results show these models can efficiently translate quantum operations into hardware-compatible gate sequences, offering a promising route to overcome current limitations and enable broader algorithmic applications.
Based on the previous three lines of research, ALEPH has made the following contributions:
On the one hand, the ALEPH project achieved major advances in applying reinforcement learning (RL) to target search problems. We demonstrated that RL agents can autonomously learn efficient search strategies across a wide variety of scenarios, without requiring prior expert knowledge or complex optimization procedures. This represents a significant shift from traditional approaches, offering researchers a simple and flexible tool to explore diverse problem settings. However, our findings also highlight that the full potential of RL-based methods is realized when complemented by domain-specific knowledge, illustrating the strength of machine learning-assisted scientific discovery. As a direct continuation of this work, we are collaborating with experimental groups to implement these RL strategies in physical systems, where additional experimental challenges such as training time constraints and real-world noise must be addressed.
In the second research line, we focused on the autonomous extraction of physical descriptors in anomalous diffusion processes using interpretable machine learning. Our work showed that models designed to create disentangled representations remain effective even when dealing with inherently stochastic data. This not only establishes a robust foundation for applying interpretable machine learning to diffusion phenomena but also paves the way for broader applications, particularly in quantum systems where randomness is intrinsic. We expect that the methods developed during the ALEPH project will form the basis for future advances in interpretable approaches within the field of quantum science.
Finally, in the area of quantum computing, we developed new generative models that bridge the gap between theoretical algorithm design and implementation on today’s quantum hardware. In particular, we demonstrated that denoising diffusion models offer key advantages over traditional Transformer- and autoregressive-based approaches. Our models can natively handle both discrete and continuous quantum gates, avoiding the need for post-processing optimizations and enabling more efficient quantum program compilation. Moreover, the flexibility of our models makes them readily applicable to other critical tasks such as quantum error correction and adapting gate compilation to different quantum hardware platforms.
On the one hand, the ALEPH project achieved major advances in applying reinforcement learning (RL) to target search problems. We demonstrated that RL agents can autonomously learn efficient search strategies across a wide variety of scenarios, without requiring prior expert knowledge or complex optimization procedures. This represents a significant shift from traditional approaches, offering researchers a simple and flexible tool to explore diverse problem settings. However, our findings also highlight that the full potential of RL-based methods is realized when complemented by domain-specific knowledge, illustrating the strength of machine learning-assisted scientific discovery. As a direct continuation of this work, we are collaborating with experimental groups to implement these RL strategies in physical systems, where additional experimental challenges such as training time constraints and real-world noise must be addressed.
In the second research line, we focused on the autonomous extraction of physical descriptors in anomalous diffusion processes using interpretable machine learning. Our work showed that models designed to create disentangled representations remain effective even when dealing with inherently stochastic data. This not only establishes a robust foundation for applying interpretable machine learning to diffusion phenomena but also paves the way for broader applications, particularly in quantum systems where randomness is intrinsic. We expect that the methods developed during the ALEPH project will form the basis for future advances in interpretable approaches within the field of quantum science.
Finally, in the area of quantum computing, we developed new generative models that bridge the gap between theoretical algorithm design and implementation on today’s quantum hardware. In particular, we demonstrated that denoising diffusion models offer key advantages over traditional Transformer- and autoregressive-based approaches. Our models can natively handle both discrete and continuous quantum gates, avoiding the need for post-processing optimizations and enabling more efficient quantum program compilation. Moreover, the flexibility of our models makes them readily applicable to other critical tasks such as quantum error correction and adapting gate compilation to different quantum hardware platforms.