Periodic Reporting for period 1 - NEUROKIT2E (Open source deep learning platform dedicated to Embedded hardware and Europe)
Periodo di rendicontazione: 2023-06-01 al 2024-05-31
Artificial intelligence (AI) has grown at an unprecedented rate in human history, becoming an essential tool in various industries. The era of "big data" has led to an explosion of information, making AI applications increasingly valuable. Access to vast amounts of data, new algorithms, and high-efficiency graphic processors have significantly contributed to the new AI boom, which started around 2010.
AI has wide-ranging implications for numerous industry sectors and is considered the latest major technological shift, similar to the advent of the computer age or the smartphone revolution. It enables more efficient data processing, better decision-making, and automation of tasks, among other benefits.
A new trend emerging within the AI landscape is the convergence of AI and the Internet of Things (IoT) into "edge AI". Edge AI involves using machine learning algorithms and connected devices for collecting, processing, analyzing data, and making decisions close to where the data is captured. This approach addresses the inherent problems of traditional cloud systems, such as high latency and lack of security, and reduces the carbon footprint.
Edge AI opens up new opportunities for AI applications, including scalable, mission-critical, and privacy-preserving products and services. Many application domains are preparing for this revolution, including transportation, field service management, finance, healthcare, manufacturing, retail, agriculture, and supply chain.
However, current AI development frameworks are not fully equipped for this paradigm shift. NEUROKIT2E is an initiative to create and promote a new open-source framework specifically for AI code generation on all embedded platforms.
This project aims to provide an open-source and sovereign platform of tools for Embedded AI with several ambitions:
1. Position EUROPE as a world market leader by providing tools capable of meeting real-time, data confidentiality, energy consumption and usability requirements.
2. Provide a platform that will integrate hardware models with neural network models to optimize the network for embedded devices, and provide a single end-to-end development chain.
3. Develop advanced compression, pruning and optimization methods to reduce model size while maintaining the performance of the original network.
4. Enable the utilization of synchronous coding (tensors) and event-driven coding (spikes) to be combined into the same network.
5. Provide tools that allow the conversion and deployment of AI application onto a large number of industrial as well as custom low-power hardware architectures.
This groundbreaking project will be based on the Aidge open-source platform developed by the French CEA LIST Institute. It will enable exploration and automatic, rapid generation of standalone code that can be exported to a variety of embedded hardware targets. It will be compatible with existing frameworks (Tensorflow, Pytorch, ONNX), and will enable performance analysis and comparisons using tools adapted to each hardware target.
The NEUROKIT2E consortium relies on 5 EU countries active in the hardware and software activities, France, Netherlands, Austria, Germany and Italy, with a balance between private and public research: 14 private companies including 5 large industries: THALES, Infineon, TTTECH, ST, DOLPHIN ; 5 RTOs (CEA, IMEC, SAL, FBK, Fraunhofer) and 6 universities (Delft, Wien, Catania, Palermo, Messina, Rostock).
are target.
AI has wide-ranging implications for numerous industry sectors and is considered the latest major technological shift, similar to the advent of the computer age or the smartphone revolution. It enables more efficient data processing, better decision-making, and automation of tasks, among other benefits.
A new trend emerging within the AI landscape is the convergence of AI and the Internet of Things (IoT) into "edge AI". Edge AI involves using machine learning algorithms and connected devices for collecting, processing, analyzing data, and making decisions close to where the data is captured. This approach addresses the inherent problems of traditional cloud systems, such as high latency and lack of security, and reduces the carbon footprint.
Edge AI opens up new opportunities for AI applications, including scalable, mission-critical, and privacy-preserving products and services. Many application domains are preparing for this revolution, including transportation, field service management, finance, healthcare, manufacturing, retail, agriculture, and supply chain.
However, current AI development frameworks are not fully equipped for this paradigm shift. NEUROKIT2E is an initiative to create and promote a new open-source framework specifically for AI code generation on all embedded platforms.
This project aims to provide an open-source and sovereign platform of tools for Embedded AI with several ambitions:
1. Position EUROPE as a world market leader by providing tools capable of meeting real-time, data confidentiality, energy consumption and usability requirements.
2. Provide a platform that will integrate hardware models with neural network models to optimize the network for embedded devices, and provide a single end-to-end development chain.
3. Develop advanced compression, pruning and optimization methods to reduce model size while maintaining the performance of the original network.
4. Enable the utilization of synchronous coding (tensors) and event-driven coding (spikes) to be combined into the same network.
5. Provide tools that allow the conversion and deployment of AI application onto a large number of industrial as well as custom low-power hardware architectures.
This groundbreaking project will be based on the Aidge open-source platform developed by the French CEA LIST Institute. It will enable exploration and automatic, rapid generation of standalone code that can be exported to a variety of embedded hardware targets. It will be compatible with existing frameworks (Tensorflow, Pytorch, ONNX), and will enable performance analysis and comparisons using tools adapted to each hardware target.
The NEUROKIT2E consortium relies on 5 EU countries active in the hardware and software activities, France, Netherlands, Austria, Germany and Italy, with a balance between private and public research: 14 private companies including 5 large industries: THALES, Infineon, TTTECH, ST, DOLPHIN ; 5 RTOs (CEA, IMEC, SAL, FBK, Fraunhofer) and 6 universities (Delft, Wien, Catania, Palermo, Messina, Rostock).
are target.
The first phase of the project laid the foundations for the study and maturation of new techniques for developing and porting deep and complex neural networks to embedded hardware.
The first stable version of the Aidge framework for exploring, optimizing and porting neural networks to embedded hardware was released in April 2024. This open source framework is hosted by the Eclipse Foundation, and will accommodate developments by project partners (some specifications have been produced). AIDGE is based on the principle of “modular architecture”, i.e. plugins with specific functionalities can be added without the need to recompile the whole framework. A 'no-code' version is currently being created, enabling the end-to-end process of training, simplifying, optimizing and exporting a network to be launched via a graphical interface.
The application part of the project is structured around six use-cases, in a wide variety of fields:
• Autonomous transport (assessment of aging and estimation of the remaining lifetime of a Li-Ion battery, autonomous people mover),
• Satellite observation of earth (artificial intelligence processing by embedded satellite for automated extraction of information from earth observation data),
• Healthcare (Service robot, to take care of frail people and help their caregivers),
• Biometrics (real-time biometrics on cell phones),
• Smart building (occupancy monitoring for efficient building control).
Specifications have been drawn up, and generic and specific KPIs have been selected.
The partners have begun to investigate and test flexible, low-precision quantization methods for weights and activations corresponding to operator implementation of hardware IP. Quantization- and hardware-aware training methods are being investigated, particularly for complex networks such as object detectors (SSD and YOLO) and transformer-based attentional mechanisms. Low-bit post-training quantization methods are also under investigation and have been tested on keyword retrieval and gesture recognition applications. Quantization to 4 bits showed very little performance loss, of the order of 2%.
Sparsification methods are also being studied, aiming at reducing the number of parameters in a network, thereby improving computational and memory efficiency. Sparsification offers advantages such as enhanced efficiency, better generalization, and increased interpretability. However, it also presents challenges, including achieving the right balance of sparsity, potential loss of information, implementation complexity, and the need to adapt techniques to different architectures and data types.
A key advantage of the AIDGE framework and the Neurokit2E project is the consideration of hardware specificities throughout the neural network implementation chain. Hardware models for simulating export targets and accelerators are studied in the project as well as mapping models that represent data traffic in an architecture, making it possible to describe the different methods of executing a DNN on a given architecture.
The first stable version of the Aidge framework for exploring, optimizing and porting neural networks to embedded hardware was released in April 2024. This open source framework is hosted by the Eclipse Foundation, and will accommodate developments by project partners (some specifications have been produced). AIDGE is based on the principle of “modular architecture”, i.e. plugins with specific functionalities can be added without the need to recompile the whole framework. A 'no-code' version is currently being created, enabling the end-to-end process of training, simplifying, optimizing and exporting a network to be launched via a graphical interface.
The application part of the project is structured around six use-cases, in a wide variety of fields:
• Autonomous transport (assessment of aging and estimation of the remaining lifetime of a Li-Ion battery, autonomous people mover),
• Satellite observation of earth (artificial intelligence processing by embedded satellite for automated extraction of information from earth observation data),
• Healthcare (Service robot, to take care of frail people and help their caregivers),
• Biometrics (real-time biometrics on cell phones),
• Smart building (occupancy monitoring for efficient building control).
Specifications have been drawn up, and generic and specific KPIs have been selected.
The partners have begun to investigate and test flexible, low-precision quantization methods for weights and activations corresponding to operator implementation of hardware IP. Quantization- and hardware-aware training methods are being investigated, particularly for complex networks such as object detectors (SSD and YOLO) and transformer-based attentional mechanisms. Low-bit post-training quantization methods are also under investigation and have been tested on keyword retrieval and gesture recognition applications. Quantization to 4 bits showed very little performance loss, of the order of 2%.
Sparsification methods are also being studied, aiming at reducing the number of parameters in a network, thereby improving computational and memory efficiency. Sparsification offers advantages such as enhanced efficiency, better generalization, and increased interpretability. However, it also presents challenges, including achieving the right balance of sparsity, potential loss of information, implementation complexity, and the need to adapt techniques to different architectures and data types.
A key advantage of the AIDGE framework and the Neurokit2E project is the consideration of hardware specificities throughout the neural network implementation chain. Hardware models for simulating export targets and accelerators are studied in the project as well as mapping models that represent data traffic in an architecture, making it possible to describe the different methods of executing a DNN on a given architecture.
Given that this project is only at its initial states, the results produced did not really go beyond the state of the art.
We produced several interesting results many of which have been published on journals or presented at conferences.
A list is available on our website: https://neurokit2e.eu/(si apre in una nuova finestra)
We produced several interesting results many of which have been published on journals or presented at conferences.
A list is available on our website: https://neurokit2e.eu/(si apre in una nuova finestra)