To enable this vision SWIMS aims to realize the information processing at the sensor node based on a fully-analog spiking pipeline – from the sensor over processing up to wireless communication – based on a neuromorphic architecture with i) as input neuron layer, new heterostructure spiking sensor arrays based on transition metal oxides/2D semiconductors for infrared (IR), ultraviolet (UV), acoustic and electromagnetic detections, ii) hidden layers in tiny spiking neural networks based on novel CMOS Fe-FET concepts capable of efficiently dealing with inherent stochastic noise when processing spiking signals on-chip, and iii) a spiking emitter as output layer for event-driven wireless transmission using optimized spike modulation. The neuromorphic processing comprises three specialized sub-networks, each serving a specific function: 1) Feature extraction from individual sensor inputs, 2) integration of extracted features from multiple sensory modalities, and 3) computation and encoding of the final decision into a spike output preceding its transmission. We assessed the feasibility of implementing these subnetworks and investigated their interaction with the neighboring spiking sensor and transmission components. We explored architectures and neural models for the feature extraction and computation networks in spiking software simulations. In parallel, we investigated whether existing neuromorphic hardware could support these networks, in particular conducting a noise analysis. As the final platform should be realized based on transition metal oxides, we established models of VO2 memristors, extended an existing model to handle stochastic processes in transition voltages, and analyzed percolation processes in VO2 memristors and the bifurcation behavior in phase-change memory-neurons. We analyzed jitter accumulation in VO2-memristor-based spiking oscillators and investigated variability and reliability of analog spiking neurons. We performed a life cycle assessment of a VO2 memristor. Lastly, we designed and taped out a physical resonate-and-fire neuron for integration into the project’s networks.
For the realization of the spiking sensors a CMOS-compatible fabrication platform for VO2 device arrays on SiO2/Si substrates, with thicknesses from 8 to 300 nm, has been established. Regarding spiking UV and IR sensors, we investigated, designed, and fabricated patterned VO2 nanostructures via atomic layer deposition (ALD) and pulsed laser deposition (PLD) to control switching conditions and stochastic behavior, highlighting the importance of in-situ annealing/melting and substrate configuration for sensing. We patterned sputtered VO2 and characterized material properties, static stochastic I–V behavior, spiking/bursting dynamics, temperature and IR sensing responses, and noise characteristics. We initiated COMSOL multiphysics modelling of VO2-based metasurfaces for IR sensing and light modulation and designed and tested IR sensors. Regarding spiking acoustic sensors we performed static pressure measurements on a silicon on insulator (SOI) micro-electromechanical system (MEMS) pressure sensor and dynamic vibration tests on Si-based MEMS membranes, targeting future integration with VO2 neurons. Towards the realization of spiking electromagnetic wave sensors we studied the adsorption spectrum of thin-film VO2 and advanced the design of spiking power detectors and metasurfaces for high frequency applications.
To communicate the decisions of the neuromorphic processor to the cloud the information is encoded in the temporal pattern of the transmitted spike train. However, the timing of the spikes is distorted by timing jitter and additive noise of the analog circuits. Moreover, the analog-to-digital (A/D) conversion at the receiver limits the temporal resolution which is critical when encoding information in time. In this context, we studied spike modulations w.r.t. to the communication performance in terms of the required energy per communicated event. Based on measurements of existing spike emitters we started to model the timing jitter of the transmitted spikes and evaluated its effect on the communication performance. Regarding energy-efficient A/D conversion we considered integrate-and-fire time encoding machines (IF-TEM) and derived a demapper providing probabilities of observed spike occurrences which can be used for subsequent decoding.
