Horizon Europe Chips JU project ViTFOX bridges Europe and South Korea to build the next generation of AI-powered chips. The Transformer deep learning architecture has been successfully applied to the natural language processing (e.g. GPT). In ViTFOX, we aim to adapt and extend the transformer architecture to computer vision. It is foreseen that vision transformers (ViT) will replace convolutional neural networks (CNN) for image recognition because they could outperform CNN with 4 times fewer computing resources
In ViTFOX, the main target is to demonstrate one of the most important components of ViT, that is an analog compute-in-memory (CIM) unit using Si-compatible Haflia-based oxide ferroelectrics. For the implementation, two different device technologies will be employed: First a ferroelectric tunnel unction (FT) memristor technology driven by the EU part of the consortium, second, a ferroelectric random access memory (FeRAM) technology driven by the Korean part.
In brief, the high-level (system level) objectives are:
-Hardware-Software co-optimization for ViT with ferroelectric oxides expecting 30% energy improvement over DRAM
-Design circuit level simulator integrating: compact models of ferroelectric oxides, CMOS and peripherals targeting energy efficiency > 50 TOPS/Watt
-Design and fabrication of CIM demonstrator with a single layer network, sense amplifiers, logic controller
Two different implementations are envisaged with the aim to obtain energy efficiency >30 TOPS/W and accuracy >90%:
-32x32 arrays with epitaxial Ferroelectric Tunnel Junction (FTJ) synaptic weights,
-32x32 arrays with 1T-4C 3D FeRAM
The FTJ-based CIM will be monolithically integrated in the back-end of line (BEOL)of CMOS. To achieve this ambitious objective, the EU and Korean parts will tightly collaborate through the whole value chain from materials and devices to advanced processing and system design. The FeRAM will be fabricated separately and will be vertically stacked with CMOS. In parallel, the consortium in a proof-of-concept activity will heterogeneously integrate novel epitaxial ferroelectric synaptic devices by wafer bonding to identify major obstacles and show performance and yield.
