Periodic Reporting for period 1 - Ekkono FLIIoT (Ekkono Synthesis: Federated Learning for the Industrial IoT)
Reporting period: 2023-05-01 to 2024-06-30
Context
The Synthesis project addresses the need for efficient, scalable, and privacy-preserving machine learning solutions in the industrial sector. With the growth of IoT and connected edge devices, industries rely on data-driven insights to optimise operations and predict maintenance needs. Traditional centralised machine learning faces challenges such as data privacy concerns, high latency, and substantial bandwidth requirements.
Overall Objectives
The primary objective of Synthesis is to develop a comprehensive model lifecycle management framework that integrates federated learning (FL) and advanced model orchestration techniques. This framework aims to create robust, high-performing machine learning models on edge devices, leveraging both edge and cloud computational capabilities for seamless operation and continuous improvement.
Project Pathway to Impact
Synthesis aims to significantly impact the industrial sector by providing a scalable and efficient centralised machine learning framework addressing key challenges like data privacy, latency, and bandwidth usage. Integrating FL, outlier / anomaly detection, clustering, and model orchestration techniques will enhance operational efficiency, reduce maintenance costs, and improve predictive capabilities. Collaborations with industrial partners in sectors like manufacturing, energy, and environmental management ensure real-world applicability and continuous refinement.
In summary, Synthesis aims to transform industrial machine learning by offering a robust, scalable, and privacy-preserving framework through strategic partnerships and real-world applications.
The Synthesis project addresses the need for efficient, scalable, and privacy-preserving machine learning solutions in the industrial sector. With the growth of IoT and connected edge devices, industries rely on data-driven insights to optimise operations and predict maintenance needs. Traditional centralised machine learning faces challenges such as data privacy concerns, high latency, and substantial bandwidth requirements.
Overall Objectives
The primary objective of Synthesis is to develop a comprehensive model lifecycle management framework that integrates federated learning (FL) and advanced model orchestration techniques. This framework aims to create robust, high-performing machine learning models on edge devices, leveraging both edge and cloud computational capabilities for seamless operation and continuous improvement.
Project Pathway to Impact
Synthesis aims to significantly impact the industrial sector by providing a scalable and efficient centralised machine learning framework addressing key challenges like data privacy, latency, and bandwidth usage. Integrating FL, outlier / anomaly detection, clustering, and model orchestration techniques will enhance operational efficiency, reduce maintenance costs, and improve predictive capabilities. Collaborations with industrial partners in sectors like manufacturing, energy, and environmental management ensure real-world applicability and continuous refinement.
In summary, Synthesis aims to transform industrial machine learning by offering a robust, scalable, and privacy-preserving framework through strategic partnerships and real-world applications.
Synthesis aims to develop a robust, scalable, and privacy-preserving model lifecycle management framework. The first year of the project has been dedicated to developing and validating core algorithms essential for this framework. Specifically, the focus has been on enhancing edge and federated learning capabilities, improving model performance, and ensuring adaptability across various industrial applications by using models from the devices instead of their data in the cloud.
1. Development of a Model Agnostic FL Technique
- Implemented a new FL technique based on knowledge distillation that supports performing FL on heterogeneous models.
2. Streaming Data Summarization:
- Developed novel methods to summarise streaming data at the edge, generating descriptive statistics to be used in the cloud for synthetic data generation and other processes.
3. Creation of Similarity Metrics
- Designed and implemented metrics to measure the distance between machine learning models, enabling clustering and anomaly detection.
4. Development of Anomaly Detection and Clustering Techniques
- Designed and implemented new advanced techniques for detecting deviations and grouping similar models efficiently.
5. Design of Cloud-Agnostic Architecture:
- Ensures the framework operates seamlessly across various cloud platforms.
6. Testing in Relevant Environments
- Successfully tested algorithms and techniques in collaboration with three large industrial companies on real operational data.
The overall main achievement has been that all core algorithms have been successfully designed, implemented, and tested in a relevant environment using operational data. Furthermore, all advancements have been achieved without sharing data from the devices, preserving data privacy and security. This approach goes beyond the current state-of-the-art while ensuring that valuable insights and optimizations are derived without compromising sensitive information.
1. Development of a Model Agnostic FL Technique
- Implemented a new FL technique based on knowledge distillation that supports performing FL on heterogeneous models.
2. Streaming Data Summarization:
- Developed novel methods to summarise streaming data at the edge, generating descriptive statistics to be used in the cloud for synthetic data generation and other processes.
3. Creation of Similarity Metrics
- Designed and implemented metrics to measure the distance between machine learning models, enabling clustering and anomaly detection.
4. Development of Anomaly Detection and Clustering Techniques
- Designed and implemented new advanced techniques for detecting deviations and grouping similar models efficiently.
5. Design of Cloud-Agnostic Architecture:
- Ensures the framework operates seamlessly across various cloud platforms.
6. Testing in Relevant Environments
- Successfully tested algorithms and techniques in collaboration with three large industrial companies on real operational data.
The overall main achievement has been that all core algorithms have been successfully designed, implemented, and tested in a relevant environment using operational data. Furthermore, all advancements have been achieved without sharing data from the devices, preserving data privacy and security. This approach goes beyond the current state-of-the-art while ensuring that valuable insights and optimizations are derived without compromising sensitive information.
The Synthesis project has achieved several significant results that extend beyond the current state-of-the-art:
1. Advanced Knowledge Distillation (KD) Technique for Federated Learning (FL):
- The most notable result is the development of a novel KD technique that enables federated learning on heterogeneous models. This technique supports the aggregation of diverse models, facilitating the creation of robust starting models and enhancing overall model performance without sharing raw data. Additionally, it improves the performance of federated learning, particularly in scenarios where models have diverged significantly.
2. Novel Anomaly Detection and Clustering Techniques:
- Development of clustering and anomaly detection techniques that do not require device data. A key here is the development of similarity metrics for machine learning models that is the basis for clustering and anomaly detection.
Potential Impacts
The results of the Synthesis project have the potential to create significant impacts across various dimensions in an industrial context:
1. Enhanced Industrial Efficiency:
- The improved performance and security of machine learning models may lead to better decision-making, optimised operations, and reduced maintenance costs in industrial settings. The ability to detect anomalies and cluster similar models could enhance predictive maintenance capabilities, ensuring continuous and reliable operation of systems.
2. Scalability and Flexibility:
- The cloud-agnostic architecture and the capability to handle heterogeneous models make the Synthesis framework highly scalable and adaptable to various industrial applications. This flexibility could allow industries to deploy advanced machine learning solutions without being tied to specific cloud providers or hardware configurations.
To ensure further success, enabling real-time response for the cloud GUI is crucial. This involves performing on-the-fly computations for dynamically selected device groups, imposing real-time computation requirements on the backend.
1. Advanced Knowledge Distillation (KD) Technique for Federated Learning (FL):
- The most notable result is the development of a novel KD technique that enables federated learning on heterogeneous models. This technique supports the aggregation of diverse models, facilitating the creation of robust starting models and enhancing overall model performance without sharing raw data. Additionally, it improves the performance of federated learning, particularly in scenarios where models have diverged significantly.
2. Novel Anomaly Detection and Clustering Techniques:
- Development of clustering and anomaly detection techniques that do not require device data. A key here is the development of similarity metrics for machine learning models that is the basis for clustering and anomaly detection.
Potential Impacts
The results of the Synthesis project have the potential to create significant impacts across various dimensions in an industrial context:
1. Enhanced Industrial Efficiency:
- The improved performance and security of machine learning models may lead to better decision-making, optimised operations, and reduced maintenance costs in industrial settings. The ability to detect anomalies and cluster similar models could enhance predictive maintenance capabilities, ensuring continuous and reliable operation of systems.
2. Scalability and Flexibility:
- The cloud-agnostic architecture and the capability to handle heterogeneous models make the Synthesis framework highly scalable and adaptable to various industrial applications. This flexibility could allow industries to deploy advanced machine learning solutions without being tied to specific cloud providers or hardware configurations.
To ensure further success, enabling real-time response for the cloud GUI is crucial. This involves performing on-the-fly computations for dynamically selected device groups, imposing real-time computation requirements on the backend.