Periodic Reporting for period 2 - TEVI (Time Encoded Voice Interfaces)
Reporting period: 2022-11-01 to 2025-04-30
The development of microelectronics has enabled the deploment of artificial intelligence in portable and wearable devices. However, limiting the power consumption is crucial to enable battery operation. This requires the design of ultra-low power circuits, in the range of hundreds of nWs. Regarding smart portable applications, speech recognition is a main topic, in special voice activity detectors (VADs). VADs continuously detect human voice in noisy environments to trigger another more complex task like full-audio conversion or keyword spotting.
This EID project focuses on MEMS integrated microphones incorporating artificial intelligence (edge computing). The conventional approach to VAD or Keyword Spotting digitizes first the input audio signal. Them Intensive digital computing is performed to either detect human voice or specific words. Although the feasibility of these architectures has been proven, a high power consumption is required, making them unsuitable for smart MEMS microphones that operate in devices with limited battery life. Time-encoded based solutions are proposed in this research project to overcome this limitation.
The research project is divided into three different work packages, each one devoted to different processing stages, but with the major goal of being compatible towards a complete intelligent system:
1. Integration of MEMS-based microphones into the digitization stage. Direct encoding of the MEMS microphone with a pulse frequency modulated signal avoids the data conversion stage prior to VAD or keyword spotting.
2. Time-encoded feature extraction. Simplification of the conventional architecture composed of a set of band-pass filters and power estimators with ring-oscillator based filters for reduction of the silicon area and direct interface with the MEMS microphone.
3. Classification tasks using neural networks based on ring oscillators. Use of neuromorphic circuits that directly connect the neural network to the sensor, avoid pre-processing and feature extraction stages or rely on the outputs of workpackages 1 and 2.
This EID project focuses on MEMS integrated microphones incorporating artificial intelligence (edge computing). The conventional approach to VAD or Keyword Spotting digitizes first the input audio signal. Them Intensive digital computing is performed to either detect human voice or specific words. Although the feasibility of these architectures has been proven, a high power consumption is required, making them unsuitable for smart MEMS microphones that operate in devices with limited battery life. Time-encoded based solutions are proposed in this research project to overcome this limitation.
The research project is divided into three different work packages, each one devoted to different processing stages, but with the major goal of being compatible towards a complete intelligent system:
1. Integration of MEMS-based microphones into the digitization stage. Direct encoding of the MEMS microphone with a pulse frequency modulated signal avoids the data conversion stage prior to VAD or keyword spotting.
2. Time-encoded feature extraction. Simplification of the conventional architecture composed of a set of band-pass filters and power estimators with ring-oscillator based filters for reduction of the silicon area and direct interface with the MEMS microphone.
3. Classification tasks using neural networks based on ring oscillators. Use of neuromorphic circuits that directly connect the neural network to the sensor, avoid pre-processing and feature extraction stages or rely on the outputs of workpackages 1 and 2.
As a summary, the project passed in the first two years the initial research and first experimental phases, where the main novel ideas matured. Then a second experimental phase was executed, with a respin of the chips. The remaining steps have been finishing the journal and conference publications and the thesis books which are in their final steps. The technical objectives of the project have evolved over the execution, as can be expected in any research. As a final step the IP generated during the project has been disseminated among possible users. In general, there is still no clear link to a product or a particular application in the current market situation, as it was at the beggining of the project.
As main results of the project, there are some ideas that can be exploited:
1) MEMS in the loop Concept under several implementation circuits. The idea would require a redesign of the MEMS to be on par with existing solutions. However, it can also be applied to different sensors such as pressure sensors.
2) Pseudo-DCO circuits. This idea has been used to implement the band pass filters of WP2 but can also be extended to data converters, spiking neural networks and audio signal processing in general.
3) Time-based filters using variable delays. These filters can be used as general-purpose analog filters for frequency encoded signals or ADCs with customized noise shaping.
These ideas will be disseminated to other potential customers, or to companies in different fields such as RF microelectronics companies. Details of these results can be found in the publication list of the project. In total 9 intenational conference papers have been presented and 3 journral papers have been published with 3 more papers under preparation/review.
As main results of the project, there are some ideas that can be exploited:
1) MEMS in the loop Concept under several implementation circuits. The idea would require a redesign of the MEMS to be on par with existing solutions. However, it can also be applied to different sensors such as pressure sensors.
2) Pseudo-DCO circuits. This idea has been used to implement the band pass filters of WP2 but can also be extended to data converters, spiking neural networks and audio signal processing in general.
3) Time-based filters using variable delays. These filters can be used as general-purpose analog filters for frequency encoded signals or ADCs with customized noise shaping.
These ideas will be disseminated to other potential customers, or to companies in different fields such as RF microelectronics companies. Details of these results can be found in the publication list of the project. In total 9 intenational conference papers have been presented and 3 journral papers have been published with 3 more papers under preparation/review.
Work package 1: Design of readout circuits for MEMS that do not require neither biasing circuitry nor high bias voltage, incorporating the sensor into the ADC itself. The first proposals in this workpackage can be divided into two types:
a) The MEMS capacitor is used as a load in a ring-oscillator. This idea has been analysed at system level with behavioral simulations. The results were published in the international conference Austrochip 2022. It has been proved that the sensitivity of the MEMS is sufficiently high to achieve moderate resolution requirements, suitable for human-machine interfaces. Nevertheless, a limitation of this solution is the impossibility to implement differential architectures and the sensitivity to process and temperature.
b) The MEMS is included in a loop that regulates the oscillation frequency of a ring-oscillator with switched capacitor circuits. System level simulations have been performed showing proper results and a first test-chip has been fabricated. Experimental measurements will be done during the next months. It is expected to achieve an experimental resolution higher than the one simulated in the first idea with higher robustness against PVT variations. A journal publication is expected.
Work package 2: Implementation of scalable, compact and power efficient filter banks and power estimators. The basic building blocks of the filters are implemented usign time-integrators and signal handling based on asynchronous digital logic. The targeted objectives are an average power consumption lower than 100 nW per channel, and an occupied area lower than 5000 um2 per channel. A first approach at system level has been already simulated and published in Austrochip 2022. Also a first proposal of a low-pass filter has been taped out in a CMOS chip. It is expected to measure a sufficiently good filtering performance for feature extraction tasks with ultra low power consumption. A journal publication is expected with these measurements. Next, we want to implement a set of band-pass filters with power estimators compatible with spiking neural networks.
Work package 3: Using time-encoded based neuron cells for the proposal and implementation of neural networks that could be included into low-power classification stages of smart signal processing architectures, such as VADs. At this point of the research, two lines of research are being studied:
a) Proposal of neural network composed of a first layer with ring-oscillator based Multiply-Accumulate (MAC) cells followed by digital GRU units. It has been simulated for VAD tasks, with accuracy detection parameters (AUC) higher than 80% even within noisy environments. Power estimations from schematic simulations are only 24 nW for each neuron cell. These results have been submitted for publication to the 2023 ISCAS conference. A ring-oscillator based MAC cell has been taped out in a CMOS chip, under fabrication. The chip measurements will be submitted to a journal (e.g. IEEE Transactions on Circuits and Systems - II). We expect to build next a complete neural network on silicon, including ring-oscillator MACs, for complete characterization.
b) Neuromorphic approach for VAD, mimicking the performance of spiking neural networks with time-encoded based circuits. First simulations on this idea are being performed with behavioral models.
The main impact expected from this action is the contribution to making and keeping the EU at the forefront of microelectronics, artificial intelligence, and sensors with a growing financial market share.
a) The MEMS capacitor is used as a load in a ring-oscillator. This idea has been analysed at system level with behavioral simulations. The results were published in the international conference Austrochip 2022. It has been proved that the sensitivity of the MEMS is sufficiently high to achieve moderate resolution requirements, suitable for human-machine interfaces. Nevertheless, a limitation of this solution is the impossibility to implement differential architectures and the sensitivity to process and temperature.
b) The MEMS is included in a loop that regulates the oscillation frequency of a ring-oscillator with switched capacitor circuits. System level simulations have been performed showing proper results and a first test-chip has been fabricated. Experimental measurements will be done during the next months. It is expected to achieve an experimental resolution higher than the one simulated in the first idea with higher robustness against PVT variations. A journal publication is expected.
Work package 2: Implementation of scalable, compact and power efficient filter banks and power estimators. The basic building blocks of the filters are implemented usign time-integrators and signal handling based on asynchronous digital logic. The targeted objectives are an average power consumption lower than 100 nW per channel, and an occupied area lower than 5000 um2 per channel. A first approach at system level has been already simulated and published in Austrochip 2022. Also a first proposal of a low-pass filter has been taped out in a CMOS chip. It is expected to measure a sufficiently good filtering performance for feature extraction tasks with ultra low power consumption. A journal publication is expected with these measurements. Next, we want to implement a set of band-pass filters with power estimators compatible with spiking neural networks.
Work package 3: Using time-encoded based neuron cells for the proposal and implementation of neural networks that could be included into low-power classification stages of smart signal processing architectures, such as VADs. At this point of the research, two lines of research are being studied:
a) Proposal of neural network composed of a first layer with ring-oscillator based Multiply-Accumulate (MAC) cells followed by digital GRU units. It has been simulated for VAD tasks, with accuracy detection parameters (AUC) higher than 80% even within noisy environments. Power estimations from schematic simulations are only 24 nW for each neuron cell. These results have been submitted for publication to the 2023 ISCAS conference. A ring-oscillator based MAC cell has been taped out in a CMOS chip, under fabrication. The chip measurements will be submitted to a journal (e.g. IEEE Transactions on Circuits and Systems - II). We expect to build next a complete neural network on silicon, including ring-oscillator MACs, for complete characterization.
b) Neuromorphic approach for VAD, mimicking the performance of spiking neural networks with time-encoded based circuits. First simulations on this idea are being performed with behavioral models.
The main impact expected from this action is the contribution to making and keeping the EU at the forefront of microelectronics, artificial intelligence, and sensors with a growing financial market share.