Periodic Reporting for period 1 - SWIMS (Stochastic Spiking Wireless Multimodal Sensory Systems)
Berichtszeitraum: 2024-01-01 bis 2025-06-30
The rapid expansion of Internet of Things devices is placing significant pressure on energy resources. Traditional sensor systems suffer from energy-intensive operation of sensing, processing, as well as communications. Within this context the ERC SYNERGY SWIMS project seeks to facilitate the deployment of a new biology-inspired approach, aiming to dramatically reduce the power consumption. SWIMS is inspired by the neural system of a bee and its spike signals, computation, and sensing. It shall result in unique, energy-efficient multi-modal sensor nodes and their wireless connectivity to the cloud. Through collaborative efforts, SWIMS endeavors to present demonstrators with energy consumption reduced by over 100 times compared to current standards, representing a substantial advancement in sensor system design and functionality.
To enable this vision SWIMS aims to realize the information processing at the sensor node based on a fully-analog spiking pipeline – from the sensor over processing up to wireless communication – based on a neuromorphic architecture with i) as input neuron layer, new heterostructure spiking sensor arrays based on transition metal oxides/2D semiconductors for infrared (IR), ultraviolet (UV), acoustic and electromagnetic detections, ii) hidden layers in tiny spiking neural networks based on novel CMOS Fe-FET concepts capable of efficiently dealing with inherent stochastic noise when processing spiking signals on-chip, and iii) a spiking emitter as output layer for event-driven wireless transmission using optimized spike modulation. The neuromorphic processing comprises three specialized sub-networks, each serving a specific function: 1) Feature extraction from individual sensor inputs, 2) integration of extracted features from multiple sensory modalities, and 3) computation and encoding of the final decision into a spike output preceding its transmission. We assessed the feasibility of implementing these subnetworks and investigated their interaction with the neighboring spiking sensor and transmission components. We explored architectures and neural models for the feature extraction and computation networks in spiking software simulations. In parallel, we investigated whether existing neuromorphic hardware could support these networks, in particular conducting a noise analysis. As the final platform should be realized based on transition metal oxides, we established models of VO2 memristors, extended an existing model to handle stochastic processes in transition voltages, and analyzed percolation processes in VO2 memristors and the bifurcation behavior in phase-change memory-neurons. We analyzed jitter accumulation in VO2-memristor-based spiking oscillators and investigated variability and reliability of analog spiking neurons. We performed a life cycle assessment of a VO2 memristor. Lastly, we designed and taped out a physical resonate-and-fire neuron for integration into the project’s networks.
For the realization of the spiking sensors a CMOS-compatible fabrication platform for VO2 device arrays on SiO2/Si substrates, with thicknesses from 8 to 300 nm, has been established. Regarding spiking UV and IR sensors, we investigated, designed, and fabricated patterned VO2 nanostructures via atomic layer deposition (ALD) and pulsed laser deposition (PLD) to control switching conditions and stochastic behavior, highlighting the importance of in-situ annealing/melting and substrate configuration for sensing. We patterned sputtered VO2 and characterized material properties, static stochastic I–V behavior, spiking/bursting dynamics, temperature and IR sensing responses, and noise characteristics. We initiated COMSOL multiphysics modelling of VO2-based metasurfaces for IR sensing and light modulation and designed and tested IR sensors. Regarding spiking acoustic sensors we performed static pressure measurements on a silicon on insulator (SOI) micro-electromechanical system (MEMS) pressure sensor and dynamic vibration tests on Si-based MEMS membranes, targeting future integration with VO2 neurons. Towards the realization of spiking electromagnetic wave sensors we studied the adsorption spectrum of thin-film VO2 and advanced the design of spiking power detectors and metasurfaces for high frequency applications.
To communicate the decisions of the neuromorphic processor to the cloud the information is encoded in the temporal pattern of the transmitted spike train. However, the timing of the spikes is distorted by timing jitter and additive noise of the analog circuits. Moreover, the analog-to-digital (A/D) conversion at the receiver limits the temporal resolution which is critical when encoding information in time. In this context, we studied spike modulations w.r.t. to the communication performance in terms of the required energy per communicated event. Based on measurements of existing spike emitters we started to model the timing jitter of the transmitted spikes and evaluated its effect on the communication performance. Regarding energy-efficient A/D conversion we considered integrate-and-fire time encoding machines (IF-TEM) and derived a demapper providing probabilities of observed spike occurrences which can be used for subsequent decoding.
For the realization of the spiking sensors a CMOS-compatible fabrication platform for VO2 device arrays on SiO2/Si substrates, with thicknesses from 8 to 300 nm, has been established. Regarding spiking UV and IR sensors, we investigated, designed, and fabricated patterned VO2 nanostructures via atomic layer deposition (ALD) and pulsed laser deposition (PLD) to control switching conditions and stochastic behavior, highlighting the importance of in-situ annealing/melting and substrate configuration for sensing. We patterned sputtered VO2 and characterized material properties, static stochastic I–V behavior, spiking/bursting dynamics, temperature and IR sensing responses, and noise characteristics. We initiated COMSOL multiphysics modelling of VO2-based metasurfaces for IR sensing and light modulation and designed and tested IR sensors. Regarding spiking acoustic sensors we performed static pressure measurements on a silicon on insulator (SOI) micro-electromechanical system (MEMS) pressure sensor and dynamic vibration tests on Si-based MEMS membranes, targeting future integration with VO2 neurons. Towards the realization of spiking electromagnetic wave sensors we studied the adsorption spectrum of thin-film VO2 and advanced the design of spiking power detectors and metasurfaces for high frequency applications.
To communicate the decisions of the neuromorphic processor to the cloud the information is encoded in the temporal pattern of the transmitted spike train. However, the timing of the spikes is distorted by timing jitter and additive noise of the analog circuits. Moreover, the analog-to-digital (A/D) conversion at the receiver limits the temporal resolution which is critical when encoding information in time. In this context, we studied spike modulations w.r.t. to the communication performance in terms of the required energy per communicated event. Based on measurements of existing spike emitters we started to model the timing jitter of the transmitted spikes and evaluated its effect on the communication performance. Regarding energy-efficient A/D conversion we considered integrate-and-fire time encoding machines (IF-TEM) and derived a demapper providing probabilities of observed spike occurrences which can be used for subsequent decoding.
SWIMS achieved several results beyond state of the arts. On the technology side we attained PLD and ALD deposition of VO2 with controlled thickness down to 6-8 nm and optimized thermal and electrical switching, understanding of electroforming mechanisms and their control in VO2 thin film, and control of stochasticity of electrical switching in VO2 films by nanopatterning and grain size engineering.
Concerning the ferroelectric synapses and neurons, we achieved TiN/Si:HfO2/Al2O3/TiN ferroelectric stacks compatible with CMOS platforms and excellent retention, aiming at building integrated FeFET based synapses. The ongoing work on the integration of a VO2 2-terminal device with a 3-terminal FeFET is the first of its kind for an integrated 1T-1R stochastic oscillator. With taping-out a mixed-signal CMOS implementation of the resonate-and-fire neuron, which can detect spiking frequencies and is endowed with address-event representation handshaking capabilities, we constructed a building block for future biologically-inspired neuromorphic processors.
The modelling of VO2 memristor-based spiking oscillators reveals for the first time a 1/f frequency dependency and excess of noise near the transition temperature. We observed sensitivity of ultra-low voltage analog spiking neurons to process variations. By improving the circuit design and fabrication processes the new VO2 neurons reached higher temperature sensitivity and lower power consumption than previously reported VO2 neurons. The realized SOI MEMS pressure sensor, featuring one of the thinnest reported small-area membrane, reached significantly higher sensitivity than almost all reported Si pressure sensors with larger or thicker membranes, as well as sensors with similar dimensions without concentrators.
The analysis of the communication performance of spike modulations unveiled that using the phase of the spikes to provide redundancy or to encode additional information enables to significantly reduce the required transmit energy per communicated bit. Likewise the proposed spike detection based on IF-TEMs enables communication at significantly lower energy per bit in comparison to uniform sampling based 1-bit A/D conversion.
Concerning the ferroelectric synapses and neurons, we achieved TiN/Si:HfO2/Al2O3/TiN ferroelectric stacks compatible with CMOS platforms and excellent retention, aiming at building integrated FeFET based synapses. The ongoing work on the integration of a VO2 2-terminal device with a 3-terminal FeFET is the first of its kind for an integrated 1T-1R stochastic oscillator. With taping-out a mixed-signal CMOS implementation of the resonate-and-fire neuron, which can detect spiking frequencies and is endowed with address-event representation handshaking capabilities, we constructed a building block for future biologically-inspired neuromorphic processors.
The modelling of VO2 memristor-based spiking oscillators reveals for the first time a 1/f frequency dependency and excess of noise near the transition temperature. We observed sensitivity of ultra-low voltage analog spiking neurons to process variations. By improving the circuit design and fabrication processes the new VO2 neurons reached higher temperature sensitivity and lower power consumption than previously reported VO2 neurons. The realized SOI MEMS pressure sensor, featuring one of the thinnest reported small-area membrane, reached significantly higher sensitivity than almost all reported Si pressure sensors with larger or thicker membranes, as well as sensors with similar dimensions without concentrators.
The analysis of the communication performance of spike modulations unveiled that using the phase of the spikes to provide redundancy or to encode additional information enables to significantly reduce the required transmit energy per communicated bit. Likewise the proposed spike detection based on IF-TEMs enables communication at significantly lower energy per bit in comparison to uniform sampling based 1-bit A/D conversion.