Periodic Reporting for period 1 - METASPIN (Metaplastic Spintronics Synapses)
Periodo di rendicontazione: 2023-02-01 al 2024-01-31
CONTEXT: A major flaw currently faced by all artificial intelligence applications is known as catastrophic forgetting, i.e. the loss of memory of previously learned tasks upon learning new ones.
OBJECTIVES: In METASPIN we envision a radically new low-power artificial synapse technology based on spintronics nanodevices that will prevent catastrophic forgetting.
We will develop a new class of neuromorphic hardware that will use magneto-ionics to support synaptic metaplasticity, i.e. a feature inspired by the human brain based on assigning a ‘hidden value’ to the states of artificial synapses to encode how important each state is. This will make it easier or harder to reconfigure the synaptic state upon learning a new task, giving a hierarchy to previously learned information and thus preventing catastrophic forgetting.
The synaptic states will be given by the two magnetisation orientations in ferromagnets with perpendicular magnetic anisotropy, and by ferro/antiferromagnetic order in materials where the two phases coexist. In all cases, magneto-ionic gating will be used to locally modulate intrinsic magnetic properties to assign ‘hidden states’ to each synaptic state. The magneto- ionic hidden states will translate into a modulation of the switching probability between synaptic states, introducing the metaplasticity functionality. In parallel, we will develop ANNs learning schemes, adapted to our device physics and inspired by biological synaptic activity, that can learn with mitigated catastrophic forgetting.
The ultimate goal of this project is to integrate this advanced synaptic technology and learning algorithms into an ANN demonstrator to test multitask learning on proof-of-concept tasks inspired by medical AI, and assess the impact of metaplasticity in catastrophic forgetting.
OBJECTIVES: In METASPIN we envision a radically new low-power artificial synapse technology based on spintronics nanodevices that will prevent catastrophic forgetting.
We will develop a new class of neuromorphic hardware that will use magneto-ionics to support synaptic metaplasticity, i.e. a feature inspired by the human brain based on assigning a ‘hidden value’ to the states of artificial synapses to encode how important each state is. This will make it easier or harder to reconfigure the synaptic state upon learning a new task, giving a hierarchy to previously learned information and thus preventing catastrophic forgetting.
The synaptic states will be given by the two magnetisation orientations in ferromagnets with perpendicular magnetic anisotropy, and by ferro/antiferromagnetic order in materials where the two phases coexist. In all cases, magneto-ionic gating will be used to locally modulate intrinsic magnetic properties to assign ‘hidden states’ to each synaptic state. The magneto- ionic hidden states will translate into a modulation of the switching probability between synaptic states, introducing the metaplasticity functionality. In parallel, we will develop ANNs learning schemes, adapted to our device physics and inspired by biological synaptic activity, that can learn with mitigated catastrophic forgetting.
The ultimate goal of this project is to integrate this advanced synaptic technology and learning algorithms into an ANN demonstrator to test multitask learning on proof-of-concept tasks inspired by medical AI, and assess the impact of metaplasticity in catastrophic forgetting.
During the first reporting period in Work Package 1, significant progress has been made in material growth techniques, particularly in the deposition of ferromagnetic (FM), antiferromagnetic (AF), buffer layers, and oxide films with sub-nanometer thickness and high uniformity across wafer-scale areas. Characterization efforts using techniques like Time-of-Flight Secondary Ion Mass Spectrometry (ToF-SIMS) and X-ray Photoelectron Spectroscopy (XPS) have provided insights into magneto-ionic irreversible effects in devices.
In Work Package 2, the focus has been on developing high-performing magneto-ionic devices with synaptic capabilities, aiming for significant magneto-ionic effects including magnetic anisotropy modulation, reversibility, cyclability, and retention of states over time in ambient conditions. The research primarily involves Ta/CoFeB/HfO2 layers where we explored various device geometries and materials for gate electrodes, while efforts have been made to optimize lithography processes and develop a low-noise measurement system to characterize magneto-ionic devices comprehensively. This system facilitated the study of spin-reorientation transitions induced by magneto-ionic gating, assessing reversibility, cyclability, and robustness of states under different conditions. Results from these measurements informed discussions with partners in WP3 to propose device geometries and operations for implementing metaplasticity.
Within Work Package 3, efforts focused on uncertainty quantification and continual learning with Bayesian neural networks. This included conceptual and theoretical studies of synaptic consolidation using a probabilistic framework. A library for accelerator-agnostic training of SNNs was developed. Benchmarks for continual learning in SNNs were proposed and evaluated, along with an examination of the relationship between BNN training, regularization, and metaplasticity for continual learning purposes.
In Work Package 2, the focus has been on developing high-performing magneto-ionic devices with synaptic capabilities, aiming for significant magneto-ionic effects including magnetic anisotropy modulation, reversibility, cyclability, and retention of states over time in ambient conditions. The research primarily involves Ta/CoFeB/HfO2 layers where we explored various device geometries and materials for gate electrodes, while efforts have been made to optimize lithography processes and develop a low-noise measurement system to characterize magneto-ionic devices comprehensively. This system facilitated the study of spin-reorientation transitions induced by magneto-ionic gating, assessing reversibility, cyclability, and robustness of states under different conditions. Results from these measurements informed discussions with partners in WP3 to propose device geometries and operations for implementing metaplasticity.
Within Work Package 3, efforts focused on uncertainty quantification and continual learning with Bayesian neural networks. This included conceptual and theoretical studies of synaptic consolidation using a probabilistic framework. A library for accelerator-agnostic training of SNNs was developed. Benchmarks for continual learning in SNNs were proposed and evaluated, along with an examination of the relationship between BNN training, regularization, and metaplasticity for continual learning purposes.
EXPECTED IMPACTS: The impact of the innovation developed in METASPIN is a significantly improved performance of AI applications at a reduced energy cost. Currently, embedded AI systems cannot learn continuously: a medical system, for example, cannot adjust to new data arising from its actual operation on patients and has to rely on external updates. METASPIN technology will allow autonomous systems (medical, elderly care, autonomous vehicles) to never stop learning and to become significantly less power-hungry. This will serve society by bringing high-performing and new AI applications to the market, improving quality of life through digital companionship and assistance at low power. A higher energy efficiency will also contribute to reducing climate change and Europe’s external energy dependence. A precise impact assessment will be performed in WP5 to provide quantitative projections based on our results.
A major impact is already envisioned in the economy by opening a new high-performing/low-consumption AI hardware market for edge computing applications.
A more human-like AI technology that can perform continuous learning will have a particularly positive impact in applications like elderly care, smart medical monitoring, and autonomous transportation that can drastically improve the quality of life of the most vulnerable sectors of society. This high societal impact goes in hand with an enormous economic potential in an ever-growing AI industrial sector, where the USA has rapidly taken the lead in both IoT and medical applications. The development of Europe’s AI innovation potential, to which this project contributes, is therefore urgent to keep a leading place in the global technological market. Finally, our approach also tackles the most urgent societal challenge of our times, the fight against climate change, by reducing the power consumption of AI.
AI applications are currently becoming a leading source of energy consumption in Information Technology, therefore, innovative low-power AI schemes are key to a more sustainable economic model for the EU.
A major impact is already envisioned in the economy by opening a new high-performing/low-consumption AI hardware market for edge computing applications.
A more human-like AI technology that can perform continuous learning will have a particularly positive impact in applications like elderly care, smart medical monitoring, and autonomous transportation that can drastically improve the quality of life of the most vulnerable sectors of society. This high societal impact goes in hand with an enormous economic potential in an ever-growing AI industrial sector, where the USA has rapidly taken the lead in both IoT and medical applications. The development of Europe’s AI innovation potential, to which this project contributes, is therefore urgent to keep a leading place in the global technological market. Finally, our approach also tackles the most urgent societal challenge of our times, the fight against climate change, by reducing the power consumption of AI.
AI applications are currently becoming a leading source of energy consumption in Information Technology, therefore, innovative low-power AI schemes are key to a more sustainable economic model for the EU.