The AIoTwin project investigates topics in the field of Artificial Intelligence of Things (AIoT), which connects the Internet of Things (IoT) and Artificial Intelligence (AI), introducing AI into physical environments. The research component of the AIoTwin project develops a data-driven orchestration middleware for energy-efficient IoT to support machine learning workflows. We developed open-source software called the AIoTwin orchestration middleware, which addresses the technical challenges that arise when AIoT solutions are deployed in operational edge-to-cloud environments with heterogeneous devices.
The architecture for the data-driven orchestration middleware for energy-efficient IoT supporting machine learning workflows includes a “learning pipeline” for hierarchical federated learning and an “inference pipeline” capable of meeting specific quality of service constraints associated with inference requests.
The AIoTwin orchestration middleware includes the following components which are published in the AIoTwin GitHub repository
https://github.com/aiotwin(öffnet in neuem Fenster):
1. Framework for adaptive orchestration of federated learning (FL) supporting hierarchical architectures: enables ML engineers to easily set up HFL pipelines across available edge-to-cloud infrastructure, potentially spanning different administrative domains. HFL training tasks can be initiated with a predefined cost budget, such as communication or energy costs, and the orchestration tool steers the training process to ensure that global models achieve improved accuracy while pipeline costs remain within the specified budget.
2. Implementation of hierarchical federated learning (HFL) services on top of the Flower framework: extension of the popular federated learning framework to support hierarchical setups.
3. QEdgeProxy, a QoS-aware load balancer for dynamic service routing across the computing continuum: serves as an intermediary between IoT devices and inference services placed in the edge-to-cloud continuum.
Components #1 and #2 form the learning pipeline of the AIoTwin orchestration middleware. The solution is based on Kubernetes (K3s), uses the Flower framework, and integrates our original Reconfiguration Validation Algorithm (RVA) to provide a dynamic and intelligent mechanism for deploying, monitoring, and reconfiguring HFL pipelines at runtime. Extensive experiments investigate HFL pipeline reconfiguration under specified latency or energy budgets, as well as performance on the joint infrastructure.
Component #3 forms the inference pipeline of the AIoTwin orchestration middleware. QEdgeProxy provides guarantees that a high percentage of requests originating from the same IoT device are processed within a predefined latency limit or energy budget, even in a highly-dynamic environment with many IoT devices. We reported the results of an extensive evaluation experiment of QEdgeProxy, demonstrating its superiority over state-of-the-art solutions.
In addition to the previous three components of the orchestration middleware for edge environments, the following component was also designed and implemented:
4. Simulator for decentralised training of Spatio-Temporal Graph Neural Networks (ST-GNNs): used to evaluate various distributed ST-GNN training strategies in traffic forecasting scenarios for decentralised prediction of vehicle speed in the context of a Smart City.
