Periodic Reporting for period 2 - HYBRAIN (Hybrid electronic-photonic architectures for brain-inspired computing)
Reporting period: 2023-05-01 to 2024-10-31
Cloud computing is an excellent solution to keep data and computer processing at a distance. Starting working locally again is crucial to avoid delays. However, moving heavy computing power to the local application is problematic mainly due to the data traffic between the processor and memory. Computers inspired by the human brain could form a solution to this issue. The EU-funded HYBRAIN will develop one based on ‘ultra-fast response’ technologies. Researchers will combine a number of highly innovative solutions based on integrating complementary technology platforms.
As artificial intelligence (AI) proliferates, hardware systems that can perform inference at ultralow latency, high precision and low power are crucial and urgently required to deal – especially quasi-locally, i.e. ‘in the edge’ – with massive and heterogenous data, respond in real time and avoid unintended consequences and function in complex and often unpredictable environments. Conventional digital electronics and the associated computer architecture is unable to meet these stringent requirements with sub-ms latency inference and a sub-10W power budget, using convolution neural networks (CNNs) on benchmarks such as ImageNet classification.
HYBRAIN’s vision is to realize a pathway for a radical new technology with ultrafast (~1 microsecond) and energy-efficient (~1 watt) edge AI inference based on a world-first, brain-inspired hybrid architecture of integrated photonics and unconventional electronics. The deeply entwined memory and processing like in the mammalian brain obviates the need to shuttle around synaptic weights. The most stringent latency bottleneck in CNNs is in the initial convolution layers. Our approach will take advantage of the ultrahigh throughput and low latency of photonic convolutional processors (PCPs) employing novel phase-change materials in these initial layers to radically speed up processing. Their output is processed using cascaded electronic linear and nonlinear classifier layers, based on memristive (phase-change memory) crossbar arrays and dopant network processing units, respectively. HYBRAIN’s science-towards-technology breakthrough brings together the world’s top research groups from academia and industry in complementary technology platforms. Each of these platforms is already highly promising, but by integrating them, HYBRAIN will have a transformative effect of overcoming existing barriers of latency and energy consumption and will enable a whole new spectrum of edge AI applications throughout society.
As artificial intelligence (AI) proliferates, hardware systems that can perform inference at ultralow latency, high precision and low power are crucial and urgently required to deal – especially quasi-locally, i.e. ‘in the edge’ – with massive and heterogenous data, respond in real time and avoid unintended consequences and function in complex and often unpredictable environments. Conventional digital electronics and the associated computer architecture is unable to meet these stringent requirements with sub-ms latency inference and a sub-10W power budget, using convolution neural networks (CNNs) on benchmarks such as ImageNet classification.
HYBRAIN’s vision is to realize a pathway for a radical new technology with ultrafast (~1 microsecond) and energy-efficient (~1 watt) edge AI inference based on a world-first, brain-inspired hybrid architecture of integrated photonics and unconventional electronics. The deeply entwined memory and processing like in the mammalian brain obviates the need to shuttle around synaptic weights. The most stringent latency bottleneck in CNNs is in the initial convolution layers. Our approach will take advantage of the ultrahigh throughput and low latency of photonic convolutional processors (PCPs) employing novel phase-change materials in these initial layers to radically speed up processing. Their output is processed using cascaded electronic linear and nonlinear classifier layers, based on memristive (phase-change memory) crossbar arrays and dopant network processing units, respectively. HYBRAIN’s science-towards-technology breakthrough brings together the world’s top research groups from academia and industry in complementary technology platforms. Each of these platforms is already highly promising, but by integrating them, HYBRAIN will have a transformative effect of overcoming existing barriers of latency and energy consumption and will enable a whole new spectrum of edge AI applications throughout society.
The HYBRAIN project has made significant progress toward developing a hybrid electronic-photonic brain-inspired architecture for ultra-low-latency AI edge inference applications. Key advancements include the demonstration of photonic analogue computing for feature extraction using electro-absorption modulators (EAMs), progress on interfacing an analogue in-memory computing (AIMC) chip with a photonic convolution processor (PCP) and an AIMC chip with dopant network processing units (DNPUs). Also, significant progress was made in the further understanding of the DNPU room-temperature working mechanism, and the development of prototype printed circuit boards for I/O control. Additionally, the project has successfully realized programmable photonic cells, developed control interfaces for efficient programming, and designed a system architecture to harmonize the diverse timescales of the technologies involved. These efforts are setting the stage for an integrated, ultra-low latency AI inference system by the end of the project, where we combine PCP, AIMC and DNPU circuitry in a meaningful way.
The HYBRAIN team successfully fabricated a 16×16 SiN-based crossbar array and implemented a 9×3 optical matrix with programmable weight cells using EAMs, achieving 8-bit precision for accurate edge detection in matrix-vector multiplication (MVM). We advanced wavelength-division multiplexing (WDM) for AI applications on different chip platforms, leveraging both EAMs and PCMs. In AI hardware, we developed methodologies to optimize linear analog in-memory computing (AIMC) layers, showcasing enhanced convolutional and fully connected layer performance through innovative algorithms and architectures. Collaborative demonstrations integrated AIMC with other platforms (DNPU for speech recognition and PCP for vision tasks), emphasizing hybrid system capabilities. Additionally, benchmarking tools and simulators were developed, including contributions to IBM AIHWKit. Fast operation of DNPUs was demonstrated, with latency estimations limited by the current experimental setup, suggesting untapped performance potential. Exploration of phase-change alloys (e.g. GST) for programmable electronic integration and strategies for optical hardware interfacing, including all-optical integrators for time-domain multiplication and accumulation, highlight significant advances in HYBRAIN's key technologies.
HYBRAIN will deliver substantial impact in key areas of societal need by delivering disruptive solutions to high performance information processing in AI applications, in particular for Edge Computing. The transformative goal of achieving ultralow latency in AI inference applications will lead to technology leaps in key European industries, such as automotive, data centres, cyber-security and AI-assisted health applications. In these areas, disruptive technology is crucial for satisfying exponentially growing processing demand. The HYBRAIN project will deliver this capability with long-lasting impact by merging world-leading computing architectures in a highly scalable fashion.
The HYBRAIN project merges best-in-class technologies (ultrafast photonic processing, ultralow latency analog electronic processing and high bandwidth nonlinear classification) in a revolutionary platform. For the first time, the project promises to deliver long-term scalable computation power with ultralow latency below 1 μs and, importantly, the potential to significantly improve performance metrics by industrial scaling. With a world-leading industrial player (IBM) as project partner, the HYBRAIN project is uniquely placed to deliver an application-ready solution to AI applications where current technology is failing. While today leading AI manufacturers are investing in linear improvements of computing architectures through current technology, the HYBRAIN project instead provides innovation in computing platforms. Photonic technologies are on the rise as hardware accelerators for AI applications with numerous startup companies contributing to their development, highlighting the disruptive potential for game-changing technology. Within HYBRAIN, a photonic approach is achieving exactly this functionality by removing key barriers in high-throughput CNNs. HYBRAIN thus prominently supports the establishment of a European ecosystem for hybrid post-von-Neumann computation with its hybrid platform.
HYBRAIN will deliver substantial impact in key areas of societal need by delivering disruptive solutions to high performance information processing in AI applications, in particular for Edge Computing. The transformative goal of achieving ultralow latency in AI inference applications will lead to technology leaps in key European industries, such as automotive, data centres, cyber-security and AI-assisted health applications. In these areas, disruptive technology is crucial for satisfying exponentially growing processing demand. The HYBRAIN project will deliver this capability with long-lasting impact by merging world-leading computing architectures in a highly scalable fashion.
The HYBRAIN project merges best-in-class technologies (ultrafast photonic processing, ultralow latency analog electronic processing and high bandwidth nonlinear classification) in a revolutionary platform. For the first time, the project promises to deliver long-term scalable computation power with ultralow latency below 1 μs and, importantly, the potential to significantly improve performance metrics by industrial scaling. With a world-leading industrial player (IBM) as project partner, the HYBRAIN project is uniquely placed to deliver an application-ready solution to AI applications where current technology is failing. While today leading AI manufacturers are investing in linear improvements of computing architectures through current technology, the HYBRAIN project instead provides innovation in computing platforms. Photonic technologies are on the rise as hardware accelerators for AI applications with numerous startup companies contributing to their development, highlighting the disruptive potential for game-changing technology. Within HYBRAIN, a photonic approach is achieving exactly this functionality by removing key barriers in high-throughput CNNs. HYBRAIN thus prominently supports the establishment of a European ecosystem for hybrid post-von-Neumann computation with its hybrid platform.