Periodic Reporting for period 1 - ENACT (Adaptive Scheduling and Deployments of Data Intensive Workloads on Energy Efficient Edge to Cloud Continuum)
Période du rapport: 2024-01-01 au 2025-06-30
ENACT develops cutting-edge techniques and technology solutions to realise a Cognitive Computing Continuum (CCC) that can address the needs for optimal (edge and Cloud) resource management and dynamic scaling, elasticity, and portability of hyper-distributed data-intensive applications. At infrastructure level, the project brings visibility to distributed edge and Cloud resources by developing Dynamic Graph Models capable of capturing and visualising the real-time and historic status information, connectivity types, dependencies, energy consumption etc. from diverse edge and Cloud resources. The graph models are used by AI (Graph Neural Networks - GNN) models and Deep Reinforcement Learning (DRL) agents to suggest the optimal deployment configurations for hyper distributed applications considering their specific needs. The AI (GNN and DRL) models are packaged as an intelligent decision-making engine that can replace the scheduling component of open-source solutions such as KubeEdge. This will enable real-time and predictive management of distributed infrastructure and applications. To take full advantage of the potential (compute, storage, energy efficiency etc) opportunities in the CCC, ENACT will develop an innovative Application Programming Model (APM). The APM will support the development of distributed platform agnostic applications, capable of self-determining their optimal deployment and optimal execution configurations while taking advantage of diverse resources in the CCC. An SDK to develop APM-based distributed applications will be developed. Moreover, services for automatic (zero-touch provisioning-based) resource configuration and (telemetry) data collections are developed to help design and update dynamic graph models. ENACT CCC solutions will be validated in 3 use-cases with challenging resource and application requirements.
Although several technologies are applied for decentralised computing, this concept remains not an easy task. The cloud-edge integration challenge refers not only to the diversity of different type of geographically dispersed infrastructure, but also heterogeneity of applications and data to be processed both in dynamic and pre-specified deployment configurations; see for instance common design principles for European Common Data Space through the signature of the Data Space Business Alliance. The key challenge remains on how the distributed (computational and networking) resources can be orchestrated and utilised to execute applications and to process data in a decentralised way. Current market solutions are quite advanced in cloud orchestration (where various functional and non-functional aspects are considered) but there are no proven tools and technologies that can support automatic deployments, elasticity and secure adaptability of modern data-intensive applications from edge to cloud in diverse user and application-specific contexts. Moreover, the use of AI for Cloud-to-edge orchestration is facing the challenges of producing dynamic, yet reliable deployment plans in the face of continuously changing connectivity, energy consumption and management requirement in the physical world; to enable secure communication across all parts of a compute continuum; to provide explainable evaluation of deployment decisions; to automate processes for distribution of resources and to apply dynamic load-balancing and elasticity concepts to be able to update and maintain software in the distributed infrastructure. Addressing these challenges requires integration and reliable management of edge-to-cloud resources and a Cognitive Scheduler capable of fulfilling the computing needs of hyper-distributed applications in the CCC with all its heterogeneity, resource limitations, real-time guarantees, security, privacy and energy concerns etc.
ENACT main Objectives are:
- To provide mechanisms for smartly deploy and execute distributed applications proactively based on their context, available resources, supporting the autonomous reconfiguration of resources, availability, and devices churn adjustment
- To support decentralized and proactive coordination of hyper-distributed applications strengthening transparency, openness, autonomy, and resource optimisation in novel business collaborative interactions
- To provide a toolbox to facilitate to developers the development and integration of new and existing hyper-distributed intelligent applications capable of learning from other nodes of the compute continuum
- To setup the core mechanisms to enable and boost future ENACT continuum’s adoption by multi-domain and different size companies
- To validate its tools and mechanisms in real-world scenarios that require seamless management of distributed resource, as well as efficient processing of data in hyper distributed applications
- To establish proven knowledge exchange and community building scenarios for fostering a competitive European software industry
Although several technologies are applied for decentralised computing, this concept remains not an easy task. The cloud-edge integration challenge refers not only to the diversity of different type of geographically dispersed infrastructure, but also heterogeneity of applications and data to be processed both in dynamic and pre-specified deployment configurations; see for instance common design principles for European Common Data Space through the signature of the Data Space Business Alliance. The key challenge remains on how the distributed (computational and networking) resources can be orchestrated and utilised to execute applications and to process data in a decentralised way. Current market solutions are quite advanced in cloud orchestration (where various functional and non-functional aspects are considered) but there are no proven tools and technologies that can support automatic deployments, elasticity and secure adaptability of modern data-intensive applications from edge to cloud in diverse user and application-specific contexts. Moreover, the use of AI for Cloud-to-edge orchestration is facing the challenges of producing dynamic, yet reliable deployment plans in the face of continuously changing connectivity, energy consumption and management requirement in the physical world; to enable secure communication across all parts of a compute continuum; to provide explainable evaluation of deployment decisions; to automate processes for distribution of resources and to apply dynamic load-balancing and elasticity concepts to be able to update and maintain software in the distributed infrastructure. Addressing these challenges requires integration and reliable management of edge-to-cloud resources and a Cognitive Scheduler capable of fulfilling the computing needs of hyper-distributed applications in the CCC with all its heterogeneity, resource limitations, real-time guarantees, security, privacy and energy concerns etc.
ENACT main Objectives are:
- To provide mechanisms for smartly deploy and execute distributed applications proactively based on their context, available resources, supporting the autonomous reconfiguration of resources, availability, and devices churn adjustment
- To support decentralized and proactive coordination of hyper-distributed applications strengthening transparency, openness, autonomy, and resource optimisation in novel business collaborative interactions
- To provide a toolbox to facilitate to developers the development and integration of new and existing hyper-distributed intelligent applications capable of learning from other nodes of the compute continuum
- To setup the core mechanisms to enable and boost future ENACT continuum’s adoption by multi-domain and different size companies
- To validate its tools and mechanisms in real-world scenarios that require seamless management of distributed resource, as well as efficient processing of data in hyper distributed applications
- To establish proven knowledge exchange and community building scenarios for fostering a competitive European software industry
Main Activities:
- Research activities related to CCC and design of ENACT CCC
- Development activities related to core AI Mechanisms of ENACT CCC
- Development activities related to application development, orchestration and deployment
- Development activities related to data management, security and AI trustworthiness
- Open Source Availability and corresponding actions
Main Achievements:
- State-of-the-art study regarding Computing Continuums and Definition of ENACT Vision beyond State-of-the-Art
- Definition of ENACT CCC Requirements and Specifications
- Design of ENACT CCC Architecture
- Pilot Cases Definition
- Infrastructure Analysis and setup of ENACT Development and Pilot Experimentation Infrastructure
- AI forecasting algorithms for latency and energy using features like CPU usage, memory used etc.
- DRL + GNN Agent for Optimal Orchestration
- A Virtual Training Environment to support AI models training, evaluation and XAI
- A Dynamic Graph Modeler interface is available to represent nodes of ENACT CCC
- CCC Models to describe various resources and infrastructure
- Telemetry Data Collector service to collect related data from CCC nodes
- ENACT Data and Object Space first version available
- LLM-based AI Act Compliance Checker to ensure AI trustworthiness
- Security mechanisms and zero trust practices based on Kubernetes, Cilium, Network Security Policies and Security Risk Modeller
- Zero Touch Provisioning (ZTP) service available
- Application Programming Model and Application Controller Initial Version
- Fisrt version of ENACT SDK as Eclipse IDE Plugin
- ENACT Open Source Project available at Eclipse Research Labs
- Research activities related to CCC and design of ENACT CCC
- Development activities related to core AI Mechanisms of ENACT CCC
- Development activities related to application development, orchestration and deployment
- Development activities related to data management, security and AI trustworthiness
- Open Source Availability and corresponding actions
Main Achievements:
- State-of-the-art study regarding Computing Continuums and Definition of ENACT Vision beyond State-of-the-Art
- Definition of ENACT CCC Requirements and Specifications
- Design of ENACT CCC Architecture
- Pilot Cases Definition
- Infrastructure Analysis and setup of ENACT Development and Pilot Experimentation Infrastructure
- AI forecasting algorithms for latency and energy using features like CPU usage, memory used etc.
- DRL + GNN Agent for Optimal Orchestration
- A Virtual Training Environment to support AI models training, evaluation and XAI
- A Dynamic Graph Modeler interface is available to represent nodes of ENACT CCC
- CCC Models to describe various resources and infrastructure
- Telemetry Data Collector service to collect related data from CCC nodes
- ENACT Data and Object Space first version available
- LLM-based AI Act Compliance Checker to ensure AI trustworthiness
- Security mechanisms and zero trust practices based on Kubernetes, Cilium, Network Security Policies and Security Risk Modeller
- Zero Touch Provisioning (ZTP) service available
- Application Programming Model and Application Controller Initial Version
- Fisrt version of ENACT SDK as Eclipse IDE Plugin
- ENACT Open Source Project available at Eclipse Research Labs