Periodic Reporting for period 2 - NeurONN (Two-Dimensional Oscillatory Neural Networks for Energy Efficient Neuromorphic Computing)
Okres sprawozdawczy: 2021-07-01 do 2023-06-30
Neuro-inspired computing architectures are one of the leading candidates to solve complex and large-scale associative learning problems for AI applications. The two key building blocks for neuromorphic computing are the neuron and the synapse, which form the distributed computing and memory units. In the NeurONN project, we are proposing a novel neuro-inspired computing architecture where information is encoded in the “phase” of coupled oscillating neurons or oscillatory neural networks (ONN). Specifically, VO2 metal-insulator transition (MIT) devices and 2D memristors will be developed as neurons and synapses for hardware implementations. NeurONN will showcase a novel and alternative energy efficient neuromorphic computing paradigm based on energy efficient devices and architectures. Such ONN will demonstrate synchronization and coupling dynamics for establishing collective learning behaviour, in addition to desirable characteristics such as scaling, ultra-low power computation, and high computing performance. NeurONN aims to develop the first-ever ONN hardware platform (targeting two demonstrators) and an ONN design methodology toolbox covering aspects from ONN architecture design to algorithms.
Artificial Intelligence (AI) is expected to have a radical impact on the European economy. Today AI relies on high-performance hardware systems (central processing unit (CPUs) and graphic processing unit (GPUs), specialized AI accelerators and high-performance networking equipment). With the end of CMOS scaling, it will be challenging and expensive using conventional hardware to cope with the ever-expanding AI workload and complexity. New thinking is needed to improve AI performance, lower the power consumption and match to the needs of the variety of novel applications, which cannot simply exist with conventional hardware. Both future edge and embedded devices may need some learning ability to support online learning while operating with low power consumption. NeurONN contributes to this with its ultra-low-powerr capability, high energy-efficiency and its CMOS compatible approach. The European industry will benefit from the ONN technology to enable online learning on the edge based on an energy efficient and alternative neuromorphic computing paradigm. Countless industry applications are interested in AI edge computing solutions in their products based on the use of NVIDIA chips on deep learning. Healthcare, Security & Surveillance, Robotics, Mobile, Devices and Gaming, Smart Sensors are some of the sectors that can benefit from the ONN technology.
The main objectives of NeurONN are:
(1) Demonstrate novel low-power computing devices based on metal-insulating-transition VO2 devices as the basis for emulating the oscillatory behavior of neurons.
(2) Demonstrate novel low-power nonvolatile resistive switches based on 2D nanomaterials for emulating the synaptic weights.
(3) Demonstrate proof-of-concept on unconventional computing paradigm based on oscillatory neural networks (ONN) in FPGAs and silicon CMOS.
(4) Develop the integration process for enabling ONN implementation based on MIT VO2 devices coupled with 2D memristors for energy efficient neuromorphic computing.
(5) Develop the atomistic to device-level modeling to understand the technology impact on process variation, interconnects, device uniformity and performance.
(6) Develop the advanced design methods covering aspects from devices, circuits to architecture-level for design-technology solutions for reliability, performance and energy-efficiency.
(7) Exploit the participation of academic and industrial partners from different scientific backgrounds to reinforce EU Nanoelectronics industry capability, innovate and sustain the technology integration requirements on beyond CMOS and also train the young generation of scientists in cutting-edge technologies.
Artificial Intelligence (AI) is expected to have a radical impact on the European economy. Today AI relies on high-performance hardware systems (central processing unit (CPUs) and graphic processing unit (GPUs), specialized AI accelerators and high-performance networking equipment). With the end of CMOS scaling, it will be challenging and expensive using conventional hardware to cope with the ever-expanding AI workload and complexity. New thinking is needed to improve AI performance, lower the power consumption and match to the needs of the variety of novel applications, which cannot simply exist with conventional hardware. Both future edge and embedded devices may need some learning ability to support online learning while operating with low power consumption. NeurONN contributes to this with its ultra-low-powerr capability, high energy-efficiency and its CMOS compatible approach. The European industry will benefit from the ONN technology to enable online learning on the edge based on an energy efficient and alternative neuromorphic computing paradigm. Countless industry applications are interested in AI edge computing solutions in their products based on the use of NVIDIA chips on deep learning. Healthcare, Security & Surveillance, Robotics, Mobile, Devices and Gaming, Smart Sensors are some of the sectors that can benefit from the ONN technology.
The main objectives of NeurONN are:
(1) Demonstrate novel low-power computing devices based on metal-insulating-transition VO2 devices as the basis for emulating the oscillatory behavior of neurons.
(2) Demonstrate novel low-power nonvolatile resistive switches based on 2D nanomaterials for emulating the synaptic weights.
(3) Demonstrate proof-of-concept on unconventional computing paradigm based on oscillatory neural networks (ONN) in FPGAs and silicon CMOS.
(4) Develop the integration process for enabling ONN implementation based on MIT VO2 devices coupled with 2D memristors for energy efficient neuromorphic computing.
(5) Develop the atomistic to device-level modeling to understand the technology impact on process variation, interconnects, device uniformity and performance.
(6) Develop the advanced design methods covering aspects from devices, circuits to architecture-level for design-technology solutions for reliability, performance and energy-efficiency.
(7) Exploit the participation of academic and industrial partners from different scientific backgrounds to reinforce EU Nanoelectronics industry capability, innovate and sustain the technology integration requirements on beyond CMOS and also train the young generation of scientists in cutting-edge technologies.
During the first reporting period between M1 to M18, partners have made significant progress on each WP despite the delays and limitations due to COVID-19 pandemic.
The work carried out on WP1 has been on the development of planar and crossbar VO2 devices as phase change material devices for oscillators and 2D TMD devices based on MoS2 and WS2 as memristive devices for implementing synaptic coupling. In WP2, work has been carried out on the development of atomistic simulations for intrinsic electronic properties of VO2, VO2-metal electrodes, MoS2 and MoS2-metal electrodes. Work has also been carried out on TCAD modeling and simulation of VO2 and MoS2 devices to understand the joule heating phenomenon and resistive switching, with the focus on device calibration with experimental data measurements in the project.
In WP3, work has been carried out on the development numerical methods for understanding the dynamics of coupled oscillators and their phase relationships. Two different types of coupled oscillators have been studied, single-ended coupled oscillators and differential coupled oscillators. In both architectures, analog design methods are developed and synaptic weight coefficients are derived and their mapping scheme to electrical components has been developed. Both resistive and resistive-capacitive coupling schemes have been analysed.
In WP4, work has been carried out on the development of digital oscillatory network as a proof of concept of computing in phase paradigm. Digital ONN has been implemented in FPGA and tested on various tasks such as live camera streaming image recognition and robot obstacle avoidance with embedded proximity sensors. As a milestone in the project, a robot with eight proximity sensors for obstacle avoidance was developed. T
The work carried out on WP1 has been on the development of planar and crossbar VO2 devices as phase change material devices for oscillators and 2D TMD devices based on MoS2 and WS2 as memristive devices for implementing synaptic coupling. In WP2, work has been carried out on the development of atomistic simulations for intrinsic electronic properties of VO2, VO2-metal electrodes, MoS2 and MoS2-metal electrodes. Work has also been carried out on TCAD modeling and simulation of VO2 and MoS2 devices to understand the joule heating phenomenon and resistive switching, with the focus on device calibration with experimental data measurements in the project.
In WP3, work has been carried out on the development numerical methods for understanding the dynamics of coupled oscillators and their phase relationships. Two different types of coupled oscillators have been studied, single-ended coupled oscillators and differential coupled oscillators. In both architectures, analog design methods are developed and synaptic weight coefficients are derived and their mapping scheme to electrical components has been developed. Both resistive and resistive-capacitive coupling schemes have been analysed.
In WP4, work has been carried out on the development of digital oscillatory network as a proof of concept of computing in phase paradigm. Digital ONN has been implemented in FPGA and tested on various tasks such as live camera streaming image recognition and robot obstacle avoidance with embedded proximity sensors. As a milestone in the project, a robot with eight proximity sensors for obstacle avoidance was developed. T
NeurONN will bring innovation on devices, architecture, process integration and ONN applications. Here we describe each of them:
Advancement beyond State-of-the-Art on memristor devices
The VO2 material properties will be improved by engineering the ALD deposition process and implementing a flash-anneal technique to achieve better uniformity of the layers and control of composition and strain. In this project, the deposition method of 2D-material will be realized directly on the clean interface of the first metallic layer, which is different from the reported processes up to now. Metals of choice in the first layer are gold, molybdenum or tungsten.
Advancement beyond State-of-the-Art on ONN Architecture
ONN state of the art will be advanced in two different directions: hardware implementations and learning. First, a fully digital implementation will be developed. Secondly, a hardware implementation using novel devices with VO2 devices as neurons and 2D memristors as coupling elements is a significant step ahead with respect to state-of-the-art on energy efficient neuromorphic computing.
Advancement beyond State-of-the-Art on Process Integration for ONN
Co-integration of VO2 and MoS2 devices will be developed to implement a simple ONN architecture based on VO2 and 2D memristor devices.
Advancement beyond the State-of-the-Art on ONN Applications
ONN will be able to target most neural network applications. We mainly focus on the applications geared from our partner AIM in Robotics based on single or multi-layer ONN.
Advancement beyond State-of-the-Art on memristor devices
The VO2 material properties will be improved by engineering the ALD deposition process and implementing a flash-anneal technique to achieve better uniformity of the layers and control of composition and strain. In this project, the deposition method of 2D-material will be realized directly on the clean interface of the first metallic layer, which is different from the reported processes up to now. Metals of choice in the first layer are gold, molybdenum or tungsten.
Advancement beyond State-of-the-Art on ONN Architecture
ONN state of the art will be advanced in two different directions: hardware implementations and learning. First, a fully digital implementation will be developed. Secondly, a hardware implementation using novel devices with VO2 devices as neurons and 2D memristors as coupling elements is a significant step ahead with respect to state-of-the-art on energy efficient neuromorphic computing.
Advancement beyond State-of-the-Art on Process Integration for ONN
Co-integration of VO2 and MoS2 devices will be developed to implement a simple ONN architecture based on VO2 and 2D memristor devices.
Advancement beyond the State-of-the-Art on ONN Applications
ONN will be able to target most neural network applications. We mainly focus on the applications geared from our partner AIM in Robotics based on single or multi-layer ONN.