Periodic Reporting for period 1 - FVLLMONTI (Ferroelectric Vertical Low energy Low latency low volume Modules fOr Neural network Transformers In 3D)
Periodo di rendicontazione: 2021-01-01 al 2022-06-30
Current advances in neural machine translation rely on increasingly larger artificial neural networks (NNs) that are computation-intensive and energy-hungry and require server-based implementations. But sharing sensitive data over an internet connection is a time-consuming process, making difficult real-time speech translation and raising data protection and privacy concerns. The FVLLMONTI project brings together interdisciplinary expertise in nanoelectronics, unconventional logic design, reliability, and system‐level design on one side and neural machine translation and cognition sciences on the other side to enable a lightweight in-ear device that allows speech-to-speech translation without requiring internet connectivity. The main challenge is that existing 2D electronic architectures are far behind biological neural systems in terms of real-time information-processing capabilities with comparable energy consumption because they suffer from "unscalable" interconnects. The FVLLMONTI vision is to go beyond conventional CMOS technology leveraging vertical nanowire field-effect transistors (VNWFETs) to unlock the full potential of truly 3D neuromorphic computing performance and density. This is achieved through actual VNWFETs fabrication, setting up a design-technology co-optimization approach to develop 3D stacked hardware layers of NNs and a hardware-software co-optimisation approach to empower energy-efficient machine translation with minimum loss of accuracy.
A key priority at the beginning of the project was to develop appropriate operating methods in order to guarantee and sustain the quality of interactions whithin the consortium. Within the FVLLMONTI project framework, two dense and intense collaborative activities have been structured to secure the success of the two main collective efforts: The Design-Technology-Co-Optimization (DTCO) loop and the Hardware-Software (Hw-Sw) loop. With respect to the DTCO loop, unlike the conventional value chain of micro/nano-electronics where interactions and software tools between device (transistor), measurements, compact modelling and circuit activities have been settled for decades to achieve the Von Neumann machine, the FVLLMONTI value chain is in its early stages aiming to achieve an optimal neural network structure. Because the overall technology achievement towards this 3D matrix structure is the Compute Cube Neural Network (N2C2) definition, the careful settlement of DTCO loop interactions has been a priority. Several variants of the vertical Nanowire FET technology are expected in term of junctionless devices, ambipolar devices, number of stacked gates, and ferroelectric gates. During this period, a nomenclature of these variants has been established to ease the dialogue between partners. As for the Hw-Sw loop, the main aim is the co-design of hardware-software architectures, systems and applications leveraging the novel devices characterized in the DTCO loop. The goal is to devise efficient, yet flexible architectural templates, in order to enable systematic design space explorations of different configurations, array sizes, supported data representations and operations. During this period, we extended the gem5-X full-system simulator, defining extensions as loosely and tightly coupled accelerators. These are defined as behavioural modules by designing their interface and event-based behavior, considering the internal implementation of the implemented functionality as a black-box, focusing on accelerator integration. The virtual N2C2-capable systems defined in gem5-X will be able to execute the Automated Speech Recognition (ASR) and Machine Translation (ML) applications. Application-related efforts focused on resource-aware optimization, starting from state-of-the-art approaches such as quantization and pruning of low-relevance parts of Transformer models, but building on top of them to gauge the additional benefits deriving from hardware-aware (and, in particular, N2C2-aware) approaches, including sparsity considerations among others.
Data size and functionality requirements for computing are increasing, according to the expectation that hardware performance will continue to improve, irrespective of the actual implementation. This is particularly true for Edge Computing hardware enabling embedded Artificial Intelligence (e-AI) paradigms such as that targeted by FVLLMONTI. Such paradigms are necessary to warrant low-latency, secure and contextualized computation on inhomogeneous sensory data, close to the data source and powered by a limited energy source. Consequently, excellent hardware energy efficiency is pivotal. At the transistor level, energy efficiency improves as gate length decreases and especially in vertical Gate All Around (GAA) transistors, allowing the transition from 2D to truly 3D architectures. Only a few years ago, the industry considered the vertical transistor architecture to be too extreme a device modification but recently major actors (e.g. Intel, TSMC, IMEC) now regard this concept as a credible option. Currently, new computing paradigms based on emerging technologies have received little traction in industry because of lack of proof that such approaches actually work. FVLLMONTI is expected to lead to a revolutionary breakthrough in the field of neuromorphic computing, with several collateral outcomes including heterogeneous co-integration techniques, device insight, advanced device modelling, logic techniques, neural network architectures and human language technology. FVLLMONTI will deliver proof-of-concept for a practically usable computing system combining the most promising electronic device (VNWFET) - beyond the end of the roadmap - and the most promising NN cell architecture. This will encourage and stimulate larges-scale European research institutes and companies to invest in this advanced beyond-CMOS emerging technology and to apply advanced design-technology co-optimization techniques. Eventually, these beyond-CMOS technologies and associated innovative 3D NN architectures are expected to be ultra-fast and dense, and are foreseen to revolutionize information-processing and data-storage technologies. Hence, access to successful new energy-efficient information-processing paradigms is mandatory to respond to society's ever-growing demand for information and communication.