Periodic Reporting for period 2 - METASPIN (Metaplastic Spintronics Synapses)
Periodo di rendicontazione: 2024-02-01 al 2025-07-31
CONTEXT: A major flaw currently faced by all artificial intelligence applications is known as catastrophic forgetting, i.e. the loss of memory of previously learned tasks upon learning new ones.
OBJECTIVES: In METASPIN we envision a radically new low-power artificial synapse technology based on spintronics nanodevices that will prevent catastrophic forgetting.
We will develop a new class of neuromorphic hardware that will use magneto-ionics to support synaptic metaplasticity, i.e. a feature inspired by the human brain based on assigning a ‘hidden value’ to the states of artificial synapses to encode how important each state is. This will make it easier or harder to reconfigure the synaptic state upon learning a new task, giving a hierarchy to previously learned information and thus preventing catastrophic forgetting.
The synaptic states will be given by the two magnetisation orientations in ferromagnets with perpendicular magnetic anisotropy, and by ferro/antiferromagnetic order in materials where the two phases coexist. In all cases, magneto-ionic gating will be used to locally modulate intrinsic magnetic properties to assign ‘hidden states’ to each synaptic state. The magneto- ionic hidden states will translate into a modulation of the switching probability between synaptic states, introducing the metaplasticity functionality. In parallel, we will develop ANNs learning schemes, adapted to our device physics and inspired by biological synaptic activity, that can learn with mitigated catastrophic forgetting.
The ultimate goal of this project is to integrate this advanced synaptic technology and learning algorithms into an ANN demonstrator to test multitask learning on proof-of-concept tasks inspired by medical AI, and assess the impact of metaplasticity in catastrophic forgetting.
OBJECTIVES: In METASPIN we envision a radically new low-power artificial synapse technology based on spintronics nanodevices that will prevent catastrophic forgetting.
We will develop a new class of neuromorphic hardware that will use magneto-ionics to support synaptic metaplasticity, i.e. a feature inspired by the human brain based on assigning a ‘hidden value’ to the states of artificial synapses to encode how important each state is. This will make it easier or harder to reconfigure the synaptic state upon learning a new task, giving a hierarchy to previously learned information and thus preventing catastrophic forgetting.
The synaptic states will be given by the two magnetisation orientations in ferromagnets with perpendicular magnetic anisotropy, and by ferro/antiferromagnetic order in materials where the two phases coexist. In all cases, magneto-ionic gating will be used to locally modulate intrinsic magnetic properties to assign ‘hidden states’ to each synaptic state. The magneto- ionic hidden states will translate into a modulation of the switching probability between synaptic states, introducing the metaplasticity functionality. In parallel, we will develop ANNs learning schemes, adapted to our device physics and inspired by biological synaptic activity, that can learn with mitigated catastrophic forgetting.
The ultimate goal of this project is to integrate this advanced synaptic technology and learning algorithms into an ANN demonstrator to test multitask learning on proof-of-concept tasks inspired by medical AI, and assess the impact of metaplasticity in catastrophic forgetting.
During the second reporting period the following work has been conducted:
WP1 focused on growing and supplying stacks and layers for its own activities and other work packages, with partners ensuring timely delivery through sputtering and atomic layer deposition (ALD). Key progress included optimizing underlayers and dielectric interfaces to improve spin–orbit torque efficiency, magnetic anisotropy, and magneto-ionic performance. Advances were made in defining and comparing ALD processes. Extensive characterization (XPS, ToF-SIMS, Mössbauer) contributed to understanding the chemical effects responsible for magneto-ionic gating, and multiple reversible anisotropy states were demonstrated in Pt/CoFeB/HfO2 stacks.
WP2 made significant progress in developing magneto-ionic devices for neuromorphic computing. A new voltage-driven magnetic information carrier was identified, offering promise for low-power, multistate memory. We fabricated nano-devices with multiple analogue, non-volatile states enabled by magnetic anisotropy control, and introduced a magnetic-field-dependent weight-update mechanism that improves neural network learning, a functionality reminiscent of neuromodulation in biological systems.
WP3 explored multiple strategies to improve continual learning and memory retention in neural networks using METASPIN devices as the learning substrate. Our work integrated Bayesian learning with metaplasticity in Binary Neural Networks, adapting a device-physics-based metaplastic learning rule to enhance robustness against physical variations. To overcome the common but unrealistic assumption of known task identities in continual learning benchmarks, we developed an artificial neurogenesis approach in which networks start small and expand by adding neurons only when needed.
WP1 focused on growing and supplying stacks and layers for its own activities and other work packages, with partners ensuring timely delivery through sputtering and atomic layer deposition (ALD). Key progress included optimizing underlayers and dielectric interfaces to improve spin–orbit torque efficiency, magnetic anisotropy, and magneto-ionic performance. Advances were made in defining and comparing ALD processes. Extensive characterization (XPS, ToF-SIMS, Mössbauer) contributed to understanding the chemical effects responsible for magneto-ionic gating, and multiple reversible anisotropy states were demonstrated in Pt/CoFeB/HfO2 stacks.
WP2 made significant progress in developing magneto-ionic devices for neuromorphic computing. A new voltage-driven magnetic information carrier was identified, offering promise for low-power, multistate memory. We fabricated nano-devices with multiple analogue, non-volatile states enabled by magnetic anisotropy control, and introduced a magnetic-field-dependent weight-update mechanism that improves neural network learning, a functionality reminiscent of neuromodulation in biological systems.
WP3 explored multiple strategies to improve continual learning and memory retention in neural networks using METASPIN devices as the learning substrate. Our work integrated Bayesian learning with metaplasticity in Binary Neural Networks, adapting a device-physics-based metaplastic learning rule to enhance robustness against physical variations. To overcome the common but unrealistic assumption of known task identities in continual learning benchmarks, we developed an artificial neurogenesis approach in which networks start small and expand by adding neurons only when needed.
EXPECTED IMPACTS: The impact of the innovation developed in METASPIN is a significantly improved performance of AI applications at a reduced energy cost. Currently, embedded AI systems cannot learn continuously: a medical system, for example, cannot adjust to new data arising from its actual operation on patients and has to rely on external updates. METASPIN technology will allow autonomous systems (medical, elderly care, autonomous vehicles) to never stop learning and to become significantly less power-hungry. This will serve society by bringing high-performing and new AI applications to the market, improving quality of life through digital companionship and assistance at low power. A higher energy efficiency will also contribute to reducing climate change and Europe’s external energy dependence. A precise impact assessment will be performed in WP5 to provide quantitative projections based on our results.
A major impact is already envisioned in the economy by opening a new high-performing/low-consumption AI hardware market for edge computing applications.
A more human-like AI technology that can perform continuous learning will have a particularly positive impact in applications like elderly care, smart medical monitoring, and autonomous transportation that can drastically improve the quality of life of the most vulnerable sectors of society. This high societal impact goes in hand with an enormous economic potential in an ever-growing AI industrial sector, where the USA has rapidly taken the lead in both IoT and medical applications. The development of Europe’s AI innovation potential, to which this project contributes, is therefore urgent to keep a leading place in the global technological market. Finally, our approach also tackles the most urgent societal challenge of our times, the fight against climate change, by reducing the power consumption of AI.
AI applications are currently becoming a leading source of energy consumption in Information Technology, therefore, innovative low-power AI schemes are key to a more sustainable economic model for the EU.
A major impact is already envisioned in the economy by opening a new high-performing/low-consumption AI hardware market for edge computing applications.
A more human-like AI technology that can perform continuous learning will have a particularly positive impact in applications like elderly care, smart medical monitoring, and autonomous transportation that can drastically improve the quality of life of the most vulnerable sectors of society. This high societal impact goes in hand with an enormous economic potential in an ever-growing AI industrial sector, where the USA has rapidly taken the lead in both IoT and medical applications. The development of Europe’s AI innovation potential, to which this project contributes, is therefore urgent to keep a leading place in the global technological market. Finally, our approach also tackles the most urgent societal challenge of our times, the fight against climate change, by reducing the power consumption of AI.
AI applications are currently becoming a leading source of energy consumption in Information Technology, therefore, innovative low-power AI schemes are key to a more sustainable economic model for the EU.