Periodic Reporting for period 1 - EdgeAI (Edge AI Technologies for Optimised Performance Embedded Processing)
Berichtszeitraum: 2022-12-01 bis 2023-12-31
EdgeAI is a key initiative for the European digital transition towards intelligent processing solutions at the edge. EdgeAI develops new electronic components and systems, processing architectures, connectivity, software, algorithms, and middleware by combining microelectronics, AI, embedded systems, and edge computing. EdgeAI will ensure that Europe has the necessary tools, skills, and technologies to enable edge AI as a viable alternative deployment option to legacy centralised solutions, unlocking the potential of ubiquitous AI deployment, with the long-term objective of Europe taking the lead of Intelligent Edge. EdgeAI contributes to the Green Deal twin transition with a systemic, cross-sectoral approach and delivers enhanced AI-based electronic components and systems, edge processing platforms, AI frameworks and middleware. It develops methodologies to ease, advance and tailor the design of edge AI technologies by coordinating efforts across 49 of the best R&D organisations across Europe. The project partners work to demonstrate the applicability of the developed approaches in 20 demonstrators across five industrial value chains: digital industry, energy, agrifood and beverage, mobility, and digital society, considering security, trust, and energy efficiency demands inherent in each of these demonstrators. EdgeAI significantly contributes to the grand societal challenge of increasing the intelligent processing capabilities at the edge.
The EdgeAI has achieved significant achievements by finalising the deliverable D1.1 (Report on a conceptual framework for edge AI requirements and specification), which provides an overview of the edge AI development based on the emergence of multimodal edge AI implementations that yields real-time performance at the edge for various industrial sectors. These require the integration of edge AI hardware, software, AI stack building blocks and data in an Edge AI design framework for the whole system. In this context, deliverable D1.1 provides the conceptual framework used to determine the functional requirements (FRs) and the non-functional requirements (NFRs) described for each value chain (VC) and their demonstrators (VCDs) in the consecutive WP1 deliverables (D1.2 - D1.6) in the project. The FRs and NFRs, along with the KPIs and measures, are used to monitor the technology development and the evolution of the demonstrators during the project's lifetime. After the implementation by each industrial domain value chain demonstrator is concluded, the results are consolidated to highlight commonalities and differences for the industrial domains to create a roadmap for the project and open issues beyond the project's lifetime. The specifications for the several demonstrators within each VC are ready and provided in the deliverables D1.2 – D1.6 which form the basis for the cross-industrial road mapping, the edge AI architecture and system design in WP2, the circuits and modules design in WP3, and the edge AI framework development in WP4 based on edge AI technology blocks. Two PhD theses were successfully finished and defended (VC4-EDI). The expected impacts from the activities outlined in WP1, WP2, WP3, and WP4 for edge AI technologies span several key areas, focusing on hardware and software architecture, innovative approaches to dataset collection, AI explainability, optimisation techniques, and developing efficient AI-driven applications. Overall, the outcomes and achievements of WP1, WP2, WP3, and WP4 are set to significantly advance edge AI technology, focusing on efficiency, explainability, security, and optimisation. These efforts are expected to pave the way for more intelligent, efficient, and reliable edge AI applications, meeting the growing demands of various industrial use cases and demonstrators in the five value chains. Important achievements are the contributions to the book "Advancing Edge Artificial Intelligence - System Contexts", which provides valuable insight to researchers working with edge AI technologies, ML/DL engineers, IoT designers, and intelligent systems developers looking to deploy innovative solutions at the edge. Another significant achievement during this reporting period is the creation of the Edge AI pan-European technology cluster ecosystem involving the large-scale Chips JU EdgeAI project in cooperation with CLEVER, REBECCA, TRISTAN, NEUROKIT2E, LoLiPoP IoT projects, ECSEL JU ANDANTE, AI4CSM, AI4DI, TEMPO and INSECTT projects to provide a platform to exchange knowledge and ideas among experts and professionals interested in advances in edge AI circuits and device design, AI hardware architectures, industrial edge AI technologies, toolchains, and applications. This will significantly impact the European edge AI community by strengthening the ecosystem, creating an arena for cross-fertilisation and exchanging ideas, and generating new suggestions for identifying gaps and the latest technology trends.
The state-of-the-art activities related to extracting and analysing data: 1) Design a pipeline workflow that detects or recognises interesting graphical elements (e.g. boxes, lines, texts) and then combine all the detected results in the semantical integration module. 2) Adopt an end-to-end approach using the large model (e.g. vision transformer, GPT) to get the analysed results directly. 3) Exploit Large Language Models (LLMs) to extract graphical and textual features, and another separate module combines all the results to perform semantic interpretation.
Work on advanced research topics, published as a SoA - Paper: Narges Norouzi, Sveta Orlova, Daan de Geus, and Gijs Dubbelman, "ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers", accepted for publication in the proceedings of IEEE/CvF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, US, 17-21 June 2024.
Beyond SoA activities include novel AI computing architectures based on new neuromorphic approaches, including techniques to boost the micro-architecture's energy efficiency without sacrificing attack/fault robustness. Investigate Sparse Stream Semantic Registers for their applicability in accelerating the operations necessary for Spiking Neural Networks (SNNs). Exploration of hybrid digital/analogue concepts based on Resistive Sum, tailored for executing MAC operations within Fully Connected Neural Networks.
Two novel wireless node designs based on two types of LoRa solutions (sub-gigahertz and mesh 2.4GHz) including designing and developing a hybrid AI-based HW platform combining several processing units, mesh connectivity and multi-sensing IIoT devices.
Analysing explainability methods for resource-constrained hardware and explaining results for model-agnostic and model-specific techniques. Determining the main challenges in distributed hardware for explainable and interpretable decisions.
Digital twin development for synthetic data generator and train AI on edge. Simulate events based on sensor input and inference results generated at the edge.
Usage of Generative AI to optimise and augment datasets.
Use different Neural Architecture Searches (NAS) to improve the deployment of neural networks on different edge AI platforms.
Development of mechanisms for secure transfer of ML models between different devices in a federated learning context.
Work on advanced research topics, published as a SoA - Paper: Narges Norouzi, Sveta Orlova, Daan de Geus, and Gijs Dubbelman, "ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers", accepted for publication in the proceedings of IEEE/CvF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, US, 17-21 June 2024.
Beyond SoA activities include novel AI computing architectures based on new neuromorphic approaches, including techniques to boost the micro-architecture's energy efficiency without sacrificing attack/fault robustness. Investigate Sparse Stream Semantic Registers for their applicability in accelerating the operations necessary for Spiking Neural Networks (SNNs). Exploration of hybrid digital/analogue concepts based on Resistive Sum, tailored for executing MAC operations within Fully Connected Neural Networks.
Two novel wireless node designs based on two types of LoRa solutions (sub-gigahertz and mesh 2.4GHz) including designing and developing a hybrid AI-based HW platform combining several processing units, mesh connectivity and multi-sensing IIoT devices.
Analysing explainability methods for resource-constrained hardware and explaining results for model-agnostic and model-specific techniques. Determining the main challenges in distributed hardware for explainable and interpretable decisions.
Digital twin development for synthetic data generator and train AI on edge. Simulate events based on sensor input and inference results generated at the edge.
Usage of Generative AI to optimise and augment datasets.
Use different Neural Architecture Searches (NAS) to improve the deployment of neural networks on different edge AI platforms.
Development of mechanisms for secure transfer of ML models between different devices in a federated learning context.