Periodic Reporting for period 1 - AC3 (Agile and Cognitive Cloud edge Continuum management)
Reporting period: 2023-01-01 to 2024-06-30
The key objectives of the AC3 project are to design, develop, and evaluate a Cloud Edge Continuum Computing Manager (CECCM) featuring AI/ML and XAI to:
(1) Handle Life Cycle Management (LCM) of new application models (micro-service-based) by redefining SLA, predicting application profile, and describing micro-services using a semantic-aware solution, while guaranteeing the application’s SLA.
(2) Handle Cloud Edge Continuum (CEC) infrastructure, including far-edge resources, by defining novel stateful service migration, resource scaling, and energy consumption optimization, while guaranteeing balance between resource infrastructure and application SLA.
(3) Use a programmable network connecting the CEC computing nodes to enforce intra-micro-service communication Quality of Service (QoS).
In addition to the above-mentioned objectives, AC3 aims to natively integrate data management procedures, as PaaS at CECCM, compliant with Gaia-X, while relying on a federation of CEC infrastructure that embraces well-known standards (IEEE Standard for Intercloud Interoperability - SII).
(1) Handle Life Cycle Management (LCM) of new application models (micro-service-based) by redefining SLA, predicting application profile, and describing micro-services using a semantic-aware solution, while guaranteeing the application’s SLA.
(2) Handle Cloud Edge Continuum (CEC) infrastructure, including far-edge resources, by defining novel stateful service migration, resource scaling, and energy consumption optimization, while guaranteeing balance between resource infrastructure and application SLA.
(3) Use a programmable network connecting the CEC computing nodes to enforce intra-micro-service communication Quality of Service (QoS).
In addition to the above-mentioned objectives, AC3 aims to natively integrate data management procedures, as PaaS at CECCM, compliant with Gaia-X, while relying on a federation of CEC infrastructure that embraces well-known standards (IEEE Standard for Intercloud Interoperability - SII).
After 18 months of the project's lifetime, all the WPs have started and are running. Please note that WP5 has started in M12. In terms of the achievements, the project consortium has defined:
- The first version of the AC3 architecture has been specified and released, which aims to enable the deployment of a micro-service-based application on a Cloud Edge Continuum (CEC) federation of resources. Toward this objective, the AC3 architecture has been divided into a user plane, management plane, and infrastructure plane. All the components and functional blocks of the user and the management plane, which constitute the CECC Manager (CECCM) functions, have been specified and defined to address the identified functional and non-functional requirements. Furthermore, all the inter and intra-communication interfaces of the different planes have been identified and specified. We expect an evolution of the architecture according to the feedback obtained from WP3, WP4 and WP5.
- The integration of the data management as PaaS has been defined and integrated into the high-level architecture, proposing new features to ensure that the application developer can onboard its data, either hot (sensors) or cold (data space). The AC3 data management procedures employ Gaia-X and IDS approaches by adopting the data catalog and data connectors concepts.
- An extensive survey of the technological tools has been conducted with the aim of identifying the technological enablers covering the different components of the high-level architecture. The objective of the survey is to identify the components of the architecture that need to be developed from scratch and those that can use existing ones.
- A security and trust architecture has been proposed that aims to secure the AC3 architecture and introduce a trust architecture at the resource federation level to guarantee SLA and improve trust between the CECCM and the different resource providers belonging to the federation, respectively. The security approach was based on the notion of zero-trust procedures, even between the intra-components of the CECCM; while the trust features solutions based on Blockchain and smart contracts.
- A simplified business model has been proposed that showcases the capability of the AC3 project to disturb the cloud edge continuum. Indeed, the main innovation of AC3 is to separate the CECCM from the CEC infrastructure, thus allowing new actors to enter the market that may not own infrastructure. This innovation is possible by relying on the IEEE SII cloud federation concept, which has been extended to support edge and far edge resources.
- Refine the three use cases of the projects: (1) IoT and data, (2) smart monitoring using UAV, and (3) deciphering the universe – processing hundreds of TBs of astronomy data. In the refinement, we detailed the scenarios to run as well as the AC3 architectural components to showcase and a KPI refinement.
- Initial Ontology and Sematic Reasoner approach to describe the micro-service-based applications considering inputs from the developer, including data management description. The designed application descriptor will be employed to automatically deploy the application micro-services and handle their life cycle thanks to the SLA descriptor included in the descriptor.
- First approach to implement data management as PaaS by specifying the data catalogue using Piveau software, while the Data source connectors for Hot and Cold data rely on Eclipse Data Space Connector (EDC) framework and leveraging IONOS S3 services
- Initial AI/ML-based algorithms that will take place at the management plane to handle the LCM of micro-service-based applications, such as AI/ML-based application profiles, Stateful migration of containers, XAI-based resource management, etc.
- Initial enablers for the AC3 architecture, such as Monitoring, resource discovery and exposure, GUI for the application developer, etc.
- A first approach using programmable SD-WAN to interconnect computing nodes belonging to different regions and operators while exposing Northbound API to the CECCM to control the QoS assigned to the different links connecting the micro-services composing an application.
- The initial integration plane of the CECCM components has been defined and initiated.
- In terms of communication, we report a leaflet, rollup, 1 video, website, 6 press releases, and 2 newsletters.
- In terms of dissemination, we report 3 Publications, 14 Presentations in Conferences/Workshops/Sessions/Panels, 1 Organized Special Session, and 1 Organized Workshop.
- The first version of the AC3 architecture has been specified and released, which aims to enable the deployment of a micro-service-based application on a Cloud Edge Continuum (CEC) federation of resources. Toward this objective, the AC3 architecture has been divided into a user plane, management plane, and infrastructure plane. All the components and functional blocks of the user and the management plane, which constitute the CECC Manager (CECCM) functions, have been specified and defined to address the identified functional and non-functional requirements. Furthermore, all the inter and intra-communication interfaces of the different planes have been identified and specified. We expect an evolution of the architecture according to the feedback obtained from WP3, WP4 and WP5.
- The integration of the data management as PaaS has been defined and integrated into the high-level architecture, proposing new features to ensure that the application developer can onboard its data, either hot (sensors) or cold (data space). The AC3 data management procedures employ Gaia-X and IDS approaches by adopting the data catalog and data connectors concepts.
- An extensive survey of the technological tools has been conducted with the aim of identifying the technological enablers covering the different components of the high-level architecture. The objective of the survey is to identify the components of the architecture that need to be developed from scratch and those that can use existing ones.
- A security and trust architecture has been proposed that aims to secure the AC3 architecture and introduce a trust architecture at the resource federation level to guarantee SLA and improve trust between the CECCM and the different resource providers belonging to the federation, respectively. The security approach was based on the notion of zero-trust procedures, even between the intra-components of the CECCM; while the trust features solutions based on Blockchain and smart contracts.
- A simplified business model has been proposed that showcases the capability of the AC3 project to disturb the cloud edge continuum. Indeed, the main innovation of AC3 is to separate the CECCM from the CEC infrastructure, thus allowing new actors to enter the market that may not own infrastructure. This innovation is possible by relying on the IEEE SII cloud federation concept, which has been extended to support edge and far edge resources.
- Refine the three use cases of the projects: (1) IoT and data, (2) smart monitoring using UAV, and (3) deciphering the universe – processing hundreds of TBs of astronomy data. In the refinement, we detailed the scenarios to run as well as the AC3 architectural components to showcase and a KPI refinement.
- Initial Ontology and Sematic Reasoner approach to describe the micro-service-based applications considering inputs from the developer, including data management description. The designed application descriptor will be employed to automatically deploy the application micro-services and handle their life cycle thanks to the SLA descriptor included in the descriptor.
- First approach to implement data management as PaaS by specifying the data catalogue using Piveau software, while the Data source connectors for Hot and Cold data rely on Eclipse Data Space Connector (EDC) framework and leveraging IONOS S3 services
- Initial AI/ML-based algorithms that will take place at the management plane to handle the LCM of micro-service-based applications, such as AI/ML-based application profiles, Stateful migration of containers, XAI-based resource management, etc.
- Initial enablers for the AC3 architecture, such as Monitoring, resource discovery and exposure, GUI for the application developer, etc.
- A first approach using programmable SD-WAN to interconnect computing nodes belonging to different regions and operators while exposing Northbound API to the CECCM to control the QoS assigned to the different links connecting the micro-services composing an application.
- The initial integration plane of the CECCM components has been defined and initiated.
- In terms of communication, we report a leaflet, rollup, 1 video, website, 6 press releases, and 2 newsletters.
- In terms of dissemination, we report 3 Publications, 14 Presentations in Conferences/Workshops/Sessions/Panels, 1 Organized Special Session, and 1 Organized Workshop.
WP3
OSR
- Definition of the application descriptor model (incl. SLA parameters and Data info)
- GUI-assisted application descriptor composer
Data Management
- Data Catalogue – Service catalogue environment: utilizing Piveau
- Data source connectors for Hot and Cold data based on Eclipse Data Space Connector (EDC) framework and leveraging IONOS S3 services
- Several data management micro-services are defined and made available as service add-ons
AI/ML-based LCM
- Smart application deployment and lifecycle management workflow definition
- ML-model for prediction of application behaviour
- App profile model definition
- App profile mechanism structure and interfacing with monitoring and LCM
- Migration algorithms: 2 solutions
WP4
Monitoring
- Developed a mechanism that integrates tools for monitoring and resource discovery,
AI/ML-based LCM
- Developed explainable AI for fine granular resources autoscaler,
- Developed Transformer-based approaches to predict infrastructure usage
- Combination of interpretable, multi-head attention with sequence-to-sequence learners
- Prediction of the latency, explanation of infrastructure usage and detection of SLO non-compliance
- Developed a Graph Neural Network (GNN) for spatio-temporal prediction of infrastructure usage
- Developed Reinforcement Learning techniques using Soft-Actor Citric for resource management
- Developed programmable networking solutions (SD-WAN and AC3 networking operator) to interconnect CEC computing nodes (Cloud, Edge, far edge)
OSR
- Definition of the application descriptor model (incl. SLA parameters and Data info)
- GUI-assisted application descriptor composer
Data Management
- Data Catalogue – Service catalogue environment: utilizing Piveau
- Data source connectors for Hot and Cold data based on Eclipse Data Space Connector (EDC) framework and leveraging IONOS S3 services
- Several data management micro-services are defined and made available as service add-ons
AI/ML-based LCM
- Smart application deployment and lifecycle management workflow definition
- ML-model for prediction of application behaviour
- App profile model definition
- App profile mechanism structure and interfacing with monitoring and LCM
- Migration algorithms: 2 solutions
WP4
Monitoring
- Developed a mechanism that integrates tools for monitoring and resource discovery,
AI/ML-based LCM
- Developed explainable AI for fine granular resources autoscaler,
- Developed Transformer-based approaches to predict infrastructure usage
- Combination of interpretable, multi-head attention with sequence-to-sequence learners
- Prediction of the latency, explanation of infrastructure usage and detection of SLO non-compliance
- Developed a Graph Neural Network (GNN) for spatio-temporal prediction of infrastructure usage
- Developed Reinforcement Learning techniques using Soft-Actor Citric for resource management
- Developed programmable networking solutions (SD-WAN and AC3 networking operator) to interconnect CEC computing nodes (Cloud, Edge, far edge)