We have first explored nanodevices which can be used within natively intelligent memories. Hafnium oxide based memories appeared as excellent candidates for storing information, as they are compatible with current microelectronics technology, scalable, and reliable. We realized that when used in natively intelligent memories, hafnium oxide based memories may be used in regimes different from their traditional applications, with high benefits in terms of energy efficiency and reliability. In particular, they can be used in a stochastic regime, implementing naturally the random variables of Bayesian models. Hafnium oxide based memories also feature the possibility to adapt the stored value, providing an appropriate substrate for learning. We have also investigated spin electronics-based nanodevices called superparamagnetic tunnel junctions. These intrinsically stochastic devices can provide fantastic basic elements for the sampling operation, an important part of most of our natively intelligent memories designs.
Second, we designed complete natively-intelligent memories associating nanodevices and conventional transistors. This design work involved the codesign of adapted machine learning architectures, circuits and systems, while simultaneously testing the appropriateness of the ideas on real arrays of nanodevices. We focused on three different concepts:
- Bayesian machines that perform Bayesian inference
- binarized neural networks, which are specially adapted to natively intelligent memories
- and the Bayesian version of neural networks, which provides uncertainty quantification.
These designs achieve outstanding energy efficiency through three mechanisms. They collocate logic and memory and require extremely minimal data movement within the system. They rely on simplified arithmetic. Finally, they exploit nanodevices in adapted regimes, where they are not deterministic. Using a hybrid CMOS/memristor technology, we fabricated multiple demonstrators of natively intelligent memories, incorporating between 1,024 and 16,384 memristors, which demonstrated the viability of the natively intelligent memory concepts and provided high visibility to the project. The demonstrators can be programmed with proof-of-concept applications, e.g. gesture recognition, sleep cycle classification, or arrhythmia classification.
We have also proposed several approaches to provide these memories with learning features, each exploiting the device physics in a different yet highly efficient manner:
- hybrid analog/digital training of binarized networks, which exploits the analog physics of memory devices, but requires solely digital CMOS circuitry
- Bayesian learning, which exploits the intrinsic stochastic effects of memory devices to implement a sampling operation
- Equilibrium Propagation, which incorporates the physics of devices and circuits into the learning process.
The project results were disseminated in 19 peer-reviewed journal publications (including 8 Nature X publications) and 18 international conference proceedings (including A+ conferences in several fields: IEDM, ESSCIRC, NeurIPS, and DATE). They were also presented in 49 invited and keynote presentations at international conferences, schools, and workshops. The results led to several more applied follow-up projects. Our Bayesian machine demonstrator has attracted significant interest from the press and was covered, e.g. by Nature, IEEE Spectrum, and the French edition of Scientific American. The concept of natively intelligent memory, closer to brain memory than conventional computer memory, was presented in several outreach events.
