Periodic Reporting for period 3 - SPINAPSE (Creating complexity: toward atomic spin-based neural hardware)
Période du rapport: 2022-03-01 au 2023-08-31
The use of artificial intelligence (AI) in information technology is ever-present. Yet, its pervasive use and expansion into many new technologies are leading to a larger energy footprint. This is largely based on how AI is implemented: neural networks are engineered in software and run on traditional computing architecture. While this computing architecture is fast and powerful, it was not designed with energy efficiency in mind. In an effort to improve the energy efficiency, the field of neuromorphic computing strives to create hardware that is tailored for performing machine learning tasks in a more efficient manner. Yet, the idea of creating hardware that can perform artificial intelligence related to a large scientific conundrum: how do we make materials that behave like the brain? To this extent, it is important to understand how the physical behavior of materials can be linked to concepts in machine learning and neuroscience. From a fundamental standpoint, this would enable the scientific community to design better materials for “brain-inspired computing.” But it would also lead to a deeper understanding of the physics of materials in limits that have been little explored. From a societal standpoint, this may lead to new functionality in computing and sensing technologies, where materials themselves may change their properties, based on learning.
The overall objectives of this program is to explore how magnets scaled to the limit of an individual atom can be used to create brain-inspired hardware. The project takes a bottom-up approach, aimed at creating new experimental insight into these physics questions. The bottom-up approach is based on using individual atoms, and controlling how they interact with each other. The original project takes the three most pervasive neuromorphic models used today, and set out to understand how these behavior of interacting magnets can mimic those models. The objectives of the program were coupled with creating a new state of the art in atomic imaging, where the highest resolution microscope that combines magnetic imaging with the highest spatial, energy, and temporal resolutions available today.
The overall objectives of this program is to explore how magnets scaled to the limit of an individual atom can be used to create brain-inspired hardware. The project takes a bottom-up approach, aimed at creating new experimental insight into these physics questions. The bottom-up approach is based on using individual atoms, and controlling how they interact with each other. The original project takes the three most pervasive neuromorphic models used today, and set out to understand how these behavior of interacting magnets can mimic those models. The objectives of the program were coupled with creating a new state of the art in atomic imaging, where the highest resolution microscope that combines magnetic imaging with the highest spatial, energy, and temporal resolutions available today.
At the outset of this program, it was important to understand how atomic-scale spins can be interlinked to form a so-called attractor network. In neuroscience, the concept of an attractor is often equated to an associative memory. In physics, an attractor network is linked to creating an energy landscape, where the attractor is represented by a local minimum in that landscape. The machine learning problem in materials becomes two-fold in this paradigm: how does one create an energy landscape with multiple minima/attractors, and how can these minima/attractors be tuned or learn based on information? We first showed in a theoretical study that arrays of interacting atomic spins can realize two interesting energy landscapes. The first we call a spin-Q glass, based on a recent proposition of a self-induced spin glass. The idea is strongly linked to the concept of a glass, where there are many minima in the energy landscape, but without the complex disorder characteristic of such systems. The second landscape we discovered was a multi-well landscape, that mimicked the attractor networks often found in machine learning models like the Hopfield model. Shortly after this theoretical study, we discovered that the magnetic behavior of elemental neodymium could be described by this notion of a spin-Q glass. This study resolved a 50-year-long debate about the magnetic behavior of this element on the periodic table. While the discovery of the first spin-Q glass has already lead to many new physical questions, we found that the behavior of neodymium was too glassy to be used for attractor networks as a result of aging.
In a second endeavor, we developed an atomic-scale platform to realize the first atom-by-atom neural network. This was based on a discovery in 2018, where we discovered the so-called single atom orbital memory derived from a cobalt atom on black phosphorus. In this program, we found that arrays of closely placed cobalt atoms “talk” to each other, and these interactions lead to an energy landscape the identically mimics an attractor network. Using a concept of separation of time scales, we were able to show the fabrication of an atomic synapse coupled to a chain of neurons, the smallest such neural network to date and contained in one material system. Using this discovery, we showed that this new materials platform could be used to mimic a seminal model in machine learning: the Boltzmann machine. Its novelty is not only in its size scale, but also that we have found evidence of self-adaption in this material. Using this simple model system, we can strive to answer many open fundamental questions linking materials to future paradigms in brain-inspired computing.
In a second endeavor, we developed an atomic-scale platform to realize the first atom-by-atom neural network. This was based on a discovery in 2018, where we discovered the so-called single atom orbital memory derived from a cobalt atom on black phosphorus. In this program, we found that arrays of closely placed cobalt atoms “talk” to each other, and these interactions lead to an energy landscape the identically mimics an attractor network. Using a concept of separation of time scales, we were able to show the fabrication of an atomic synapse coupled to a chain of neurons, the smallest such neural network to date and contained in one material system. Using this discovery, we showed that this new materials platform could be used to mimic a seminal model in machine learning: the Boltzmann machine. Its novelty is not only in its size scale, but also that we have found evidence of self-adaption in this material. Using this simple model system, we can strive to answer many open fundamental questions linking materials to future paradigms in brain-inspired computing.
We created a new state of the art in magnetic imaging and time resolution. We were able to combine one of the highest resolution scanning tunneling microscopes in the world, and implement time-resolved methods in it. We adapted a microscope working at milliKelvin temperature, and adopted two established techniques in this system. First, we demonstrated that we could implement electron spin resonance on an individual atom. This method has been developed earlier, but we set a record, based on pushing the method to lower temperature. This enable the highest resolution studies to date, wher we can now probe the lowest frequency bands in combination with the highest spatial resolution available today. We used this method to characterize the structure of an individual hydrogenated titanium molecule, to benchmark the method and detail the structure of this molecule. Likewise, we also adapted pump-probe methodology in this microscope, which opens up the possibility for looking at the response of individual atoms to much more complex wave forms and the study of their relaxation behavior.
In the last half of this project, we aim to understand the role of spin in these systems. We want to combine using our newly time-resolved methods as well as magnetic imaging in the atomic scale systems. In this way, we can explore implemented other machine learning algorithms in atomic scale materials, as well as understanding their time-dependent behavior.
In the last half of this project, we aim to understand the role of spin in these systems. We want to combine using our newly time-resolved methods as well as magnetic imaging in the atomic scale systems. In this way, we can explore implemented other machine learning algorithms in atomic scale materials, as well as understanding their time-dependent behavior.