Periodic Reporting for period 2 - NeuroSoC (A multiprocessor system on chip with in-memory neural processing unit)
Reporting period: 2023-09-01 to 2024-08-31
The explosive growth of artificial intelligence and its movement to the edge and end devices have prompted significant research on highly energy efficient and low-latency non-von Neumann computing paradigms such as in-memory computing (IMC). As a direct response, the objective of the NeuroSoC project is to develop a flexible computing system where an analog IMC-based neural processing unit is integrated into a multi-processor functional safe and secure SoC to tackle the requirements of a wide set of edge-AI applications. The NeuroSoC approach is relying on a solid, mature, and qualified reliable Phase Change Memory technology, that will enable to create an industrially proven path answering to the level of maturity need compatible with a mass volume production and cost.
The project is based on the development of the bricks of the NeuroSoC architecture and the integration of such bricks and various components in the SoC architecture. The development of a software layer needed for the usage of the platform and the implementation of the applications is also considered, while the consortium is also working on the definition and implementation of the use case applications required for the validation and demonstration of the solution in the different fields of applications.
Objectives will be the following:
1- Computational PCM technology: Characterization and modelling of a Phase Change Memory (PCM) device developed by ST-I in FD-SOI 28nm technology to serve as the building block of the In-Memory Computing (IMC) tile.
* Programming optimization to desired analog conductance values.
* Accurate compact models’ development to facilitate the design of the IMC tiles in WP2.
* Statistical models’ development based on array-level characterization,
* Evaluation of the overall compute precision based on the conductance fluctuations and other non-idealities associated with these devices.
2 - Circuit design: Leveraging the multilevel PCM device to design an analog IMC tile.
* Definition of the unit cell and a suitable array structure.
* Design of the associated digital and analog circuits.
* Anticipate inputs from security analysis to make the resulting IMC tile more robust against side channels attacks and for improved security.
3- Architecture: Design of a modular IP implementing a Neural Processing Unit (NPU) with a hybrid digital and analog computational model.
* Demonstration of a design-time parametric IP supporting multi-tile implementations and scalable from ultra-low cost and power applications.
* Integration in a Multi-Processor System on Chip (MPSoC) system level architecture.
* Development and integration of an enhanced version of a RISC-V microprocessor core.
4 - Algorithms, Tools & Software: Development of software extensions, primitives, deployment tools and customized algorithms to fully exploit the NeuroSoC system-level architecture.
* Exploration of novel Instruction-Set Architecture (ISA) extension techniques.
* Development of vertically integrated deployment flows targeting both IMNPU-based and software-based computation.
* Mapping use-cases and investigating strategies to exploit the whole NeuroSoC system-level architecture.
5 - Applications requirements, integration, and use- cases demonstrations: Investigate edge-applications where NeuroSoC can offer a compelling advantage.
* Selection and qualification of applications.
* Benchmarking of SoA and emerging solutions.
* Proposition of an evaluation framework.
* Assessment of performances vs requirements.
The project is based on the development of the bricks of the NeuroSoC architecture and the integration of such bricks and various components in the SoC architecture. The development of a software layer needed for the usage of the platform and the implementation of the applications is also considered, while the consortium is also working on the definition and implementation of the use case applications required for the validation and demonstration of the solution in the different fields of applications.
Objectives will be the following:
1- Computational PCM technology: Characterization and modelling of a Phase Change Memory (PCM) device developed by ST-I in FD-SOI 28nm technology to serve as the building block of the In-Memory Computing (IMC) tile.
* Programming optimization to desired analog conductance values.
* Accurate compact models’ development to facilitate the design of the IMC tiles in WP2.
* Statistical models’ development based on array-level characterization,
* Evaluation of the overall compute precision based on the conductance fluctuations and other non-idealities associated with these devices.
2 - Circuit design: Leveraging the multilevel PCM device to design an analog IMC tile.
* Definition of the unit cell and a suitable array structure.
* Design of the associated digital and analog circuits.
* Anticipate inputs from security analysis to make the resulting IMC tile more robust against side channels attacks and for improved security.
3- Architecture: Design of a modular IP implementing a Neural Processing Unit (NPU) with a hybrid digital and analog computational model.
* Demonstration of a design-time parametric IP supporting multi-tile implementations and scalable from ultra-low cost and power applications.
* Integration in a Multi-Processor System on Chip (MPSoC) system level architecture.
* Development and integration of an enhanced version of a RISC-V microprocessor core.
4 - Algorithms, Tools & Software: Development of software extensions, primitives, deployment tools and customized algorithms to fully exploit the NeuroSoC system-level architecture.
* Exploration of novel Instruction-Set Architecture (ISA) extension techniques.
* Development of vertically integrated deployment flows targeting both IMNPU-based and software-based computation.
* Mapping use-cases and investigating strategies to exploit the whole NeuroSoC system-level architecture.
5 - Applications requirements, integration, and use- cases demonstrations: Investigate edge-applications where NeuroSoC can offer a compelling advantage.
* Selection and qualification of applications.
* Benchmarking of SoA and emerging solutions.
* Proposition of an evaluation framework.
* Assessment of performances vs requirements.
It the first year of the project, the focus of the work has been the definition of the hardware architecture and the various elements needed, with a special focus on the development of a PCM NVM memory capable to store multilevel values, and to perform In Memory Computing.
For the IMNPU unit, three distinct architectures, namely the router-based, cluster-based and 2D mesh-based architectures, were evaluated for their suitability for the targeted weight stationary system. More specifically, the architectures were assessed for design flexibility, scalability, performance/cost and power efficiency. Finally, the 2D mesh-based architecture was selected as a suitable candidate for NeuroSoC. Various approaches to schedule the data processing in the mesh architecture have been discussed with the conclusion that two schemes, the time scheduling approach and the data driven approach provide advantages for specific use cases. The evaluation of these approaches brought the conclusion that both approaches seem to have their advantages and disadvantages. Since the implementation of the required hardware support for both schemes is negligible, we decided to support both schemes and to provide the software tool chain with the freedom to use one or the other depending on the use case and better applicability. The initial version of the RISC-V component has been synthesized in various configurations.
Regarding the security aspects, two methods, the "Encoder-Decoder based approach" and the "Tile-by-Tile approach," have been successfully explored, showing potential security risks on the In-Memory Computing (IMC) based systems due to Side-Channel Attacks.
For the characterization of the PCM cell, a set of different type of cells have been considered, and two batches of standard and rheostatic cells have been fabricated and extensively characterized.
On these cells a multilevel programming algorithm has been implemented. With the results obtained in the first period the multilevel is working, but convergence is not always guaranteed. The algorithm and the convergence are under refinement.
The computational accuracy of the Analog In Memory Computing based on PCM , and the comparison with the digital compute engine are under evaluation.
For the design of the AIMC tile, two architectures have been considered, one main architecture and one exploratory architecture, both starting from elements of existing tiles from IBM and ST.
An RTL design of the AIMC tile, to be integrated in the overall architecture, has been realized, and it is ready to be produced for the verification of the real hardware.
For the AIMC based on SRAM a first test chip has been realized and it is currently under evaluation in terms of non idealities of the real hardware and effect of analog noise sources.
Finally a first version of the wrapper of the AIMC tiles in the main architecture has been defined.
Regarding the development of the software toolchain to be used to program the NeuroSoC system, based on the hardware architecture that has been depicted during the first year, the software stack components have been identified and connected in a proper structure, in order to allow the deployment of neural networks on the emulator first, and finally on the NeuroSoC device. In the first year the focus has been on the definition of the software architecture and the related simulations/emulation elements, taking into consideration the needs of the end user for the demonstration of efficiency and quality level requirements.
Based on the variety of applications considered, from automotive to aerospace, security and safety, the requirements of the target applications of the NeuroSoC device have been analyzed and the possibility of fulfillment of these requirements with the defined architecture has been confirmed.
For the IMNPU unit, three distinct architectures, namely the router-based, cluster-based and 2D mesh-based architectures, were evaluated for their suitability for the targeted weight stationary system. More specifically, the architectures were assessed for design flexibility, scalability, performance/cost and power efficiency. Finally, the 2D mesh-based architecture was selected as a suitable candidate for NeuroSoC. Various approaches to schedule the data processing in the mesh architecture have been discussed with the conclusion that two schemes, the time scheduling approach and the data driven approach provide advantages for specific use cases. The evaluation of these approaches brought the conclusion that both approaches seem to have their advantages and disadvantages. Since the implementation of the required hardware support for both schemes is negligible, we decided to support both schemes and to provide the software tool chain with the freedom to use one or the other depending on the use case and better applicability. The initial version of the RISC-V component has been synthesized in various configurations.
Regarding the security aspects, two methods, the "Encoder-Decoder based approach" and the "Tile-by-Tile approach," have been successfully explored, showing potential security risks on the In-Memory Computing (IMC) based systems due to Side-Channel Attacks.
For the characterization of the PCM cell, a set of different type of cells have been considered, and two batches of standard and rheostatic cells have been fabricated and extensively characterized.
On these cells a multilevel programming algorithm has been implemented. With the results obtained in the first period the multilevel is working, but convergence is not always guaranteed. The algorithm and the convergence are under refinement.
The computational accuracy of the Analog In Memory Computing based on PCM , and the comparison with the digital compute engine are under evaluation.
For the design of the AIMC tile, two architectures have been considered, one main architecture and one exploratory architecture, both starting from elements of existing tiles from IBM and ST.
An RTL design of the AIMC tile, to be integrated in the overall architecture, has been realized, and it is ready to be produced for the verification of the real hardware.
For the AIMC based on SRAM a first test chip has been realized and it is currently under evaluation in terms of non idealities of the real hardware and effect of analog noise sources.
Finally a first version of the wrapper of the AIMC tiles in the main architecture has been defined.
Regarding the development of the software toolchain to be used to program the NeuroSoC system, based on the hardware architecture that has been depicted during the first year, the software stack components have been identified and connected in a proper structure, in order to allow the deployment of neural networks on the emulator first, and finally on the NeuroSoC device. In the first year the focus has been on the definition of the software architecture and the related simulations/emulation elements, taking into consideration the needs of the end user for the demonstration of efficiency and quality level requirements.
Based on the variety of applications considered, from automotive to aerospace, security and safety, the requirements of the target applications of the NeuroSoC device have been analyzed and the possibility of fulfillment of these requirements with the defined architecture has been confirmed.