Periodic Reporting for period 1 - Swarmchestrate (Application-level Swarm-based Orchestration Across the Cloud-to-Edge Continuum)
Okres sprawozdawczy: 2024-01-01 do 2025-06-30
Collecting and analysing large amounts of data in the Cloud-to-Edge computing continuum raises new challenges. Processing all this data centrally in cloud data centres is not feasible as transferring large amounts of data to the cloud is time-consuming, expensive, degrade performance and may raise security concerns. Therefore, novel distributed computing paradigms, such as edge and fog computing emerged to support processing data closer to its origin
To overcome current limitations, Swarmchestrate develops a novel decentralised application-level orchestrator, based on the notion of self-organised interdependent Swarms. Application microservices are managed in a dynamic Orchestration Space by decentralised Orchestration Agents, governed by distributed intelligence that provides matchmaking between application requirements and resources, and supports the dynamic self-organization of Swarms. Knowledge and trust, essential for the operation of the Orchestration Space, are managed through blockchain-based trusted solutions using methods of Self-Sovereign Identities (SSI) and Distributed Identifiers (DID). End-to-end security of the overall system is assured by utilising state-of-the-art cryptographic algorithms and privacy preserving data analytics.
Novel simulation approaches are being developed to test and optimise system behavior (e.g. energy efficiency) in the early stages of development. The simulator will be further extended into a digital twin running in parallel to the physical system and improving its behaviour with predictive feedback.
The Swarmchestrateconcept will be prototyped on four real-life demonstrators from the areas of flood prevention, parking space management, video analytics and a digital twin of natural habitat.
To overcome current limitations, Swarmchestrate develops a novel decentralised application-level orchestrator, based on the notion of self-organised interdependent Swarms. Application microservices are managed in a dynamic Orchestration Space by decentralised Orchestration Agents, governed by distributed intelligence that provides matchmaking between application requirements and resources, and supports the dynamic self-organization of Swarms. Knowledge and trust, essential for the operation of the Orchestration Space, are managed through blockchain-based trusted solutions using methods of Self-Sovereign Identities (SSI) and Distributed Identifiers (DID). End-to-end security of the overall system is assured by utilising state-of-the-art cryptographic algorithms and privacy preserving data analytics.
Novel simulation approaches are being developed to test and optimise system behavior (e.g. energy efficiency) in the early stages of development. The simulator will be further extended into a digital twin running in parallel to the physical system and improving its behaviour with predictive feedback.
The Swarmchestrateconcept will be prototyped on four real-life demonstrators from the areas of flood prevention, parking space management, video analytics and a digital twin of natural habitat.
The Swarmchestrate project progressed according to its original workplan in the first reporting period. No major deviations have accured that may put the successful completion of the project into danger. Minor deviations are addressed in the Technical Report (Part B).
The major overall achievement of the project is the completion of a first prototype of a Swarmchestrate orchestrator that has been tested with a sample application. To achieve this, all technical work packages needed to significantly contribute, as detailed below.
WP1 consolidated the notion of logical proximity and it is partly implemented in the cost function for Swarm initialization.The WP also formalized logical proximity including the QoS attributes, resource descriptions, and application submission. Distribution of resource offers have been accomplished, and cost-function-based ranking functions are being integrated for the matchmaking. Finally, multiple optimization algorithms have been developed for resource ranking.
WP2 has defined the high-level structure of the TOSCA description template. Initial version of QoS parameters have been specified and agreed with other technical WPs. A JSON specification has been completed that provides an intermediary format to communicate the content of the TOSCA description with the various Swarmchestrate components. Work on a GUI that converts key/value pairs provided by application and resource owners into a TOSCA 2.0 specification has started. High-level architecture of deployment mechanism has been defined with an architecture diagram and related algorithms. Kubernetes has been selected as technology to support deployment and runtime management of applications. Implementation of essential libraries for P2P communication, VM and container handling and capacity management has been completed. Research of the current state-of-the-art with respect capable monitoring frameworks for Swarms has been done. Analysis and development of TOSCA-based application descriptions that adequately capture monitoring requirements has beecompleted. A first version of the Event Management System (EMS) extensions towards decentralisation capabilities has been implemented. An initial energy optimisation model has been defined.
WP3 completed a SOTA analysis for trust in the Things-to-Cloud continuum, defining trust in the system. It also conducted research and analysis on the Swarmchestrate platform and establishing trust in such an environment. It defined trust, outlined key trust concepts, and explored various trust attributes that contribute towards achieving trustworthiness. The WP also analysed the advantages of using Decentralized Identifiers (DIDs) and Verifiable Credentials (VCs) to establish trustworthy interactions between entities within the Swarmchestrate ecosystem, created an nintial design for the Trust Management System, implemented a preliminary prototype, incorporating direct, context-based, and reputation-based trust, and created an integration plan for embedding the system into the platform. WP3 collected analysed an defined privacy and security requirements of Swarmchestrate services and demonstrators, and started the implementation of a privacy-preserving solution for matchmaking of application requirements and resource capabilities.
In WP4 main system components and their functionalities have been defined, together with a high-level architecture of the Swarmchestrate system. Every subsystem has defined its input and output interfaces as a black-box to support integration. Main features for the demonstrators have been defined and presented in diagrams, requirements lists, scaling rules and their related monitoring metrics. System design for the Swarmchestrate demonstrators applications has been completed, followed by the prototype development of the demonstrator applications. WP4 also designed the model of the Resource Agent for the simulation environment, matching the requirements of the Swarmchestrate deployment mechanism. Finally, the WP Improved the models representing Resource Agents, Applications, and Capacities to simulate the application deployment phase, within the Cloud-to-Edge Continuum.
The major overall achievement of the project is the completion of a first prototype of a Swarmchestrate orchestrator that has been tested with a sample application. To achieve this, all technical work packages needed to significantly contribute, as detailed below.
WP1 consolidated the notion of logical proximity and it is partly implemented in the cost function for Swarm initialization.The WP also formalized logical proximity including the QoS attributes, resource descriptions, and application submission. Distribution of resource offers have been accomplished, and cost-function-based ranking functions are being integrated for the matchmaking. Finally, multiple optimization algorithms have been developed for resource ranking.
WP2 has defined the high-level structure of the TOSCA description template. Initial version of QoS parameters have been specified and agreed with other technical WPs. A JSON specification has been completed that provides an intermediary format to communicate the content of the TOSCA description with the various Swarmchestrate components. Work on a GUI that converts key/value pairs provided by application and resource owners into a TOSCA 2.0 specification has started. High-level architecture of deployment mechanism has been defined with an architecture diagram and related algorithms. Kubernetes has been selected as technology to support deployment and runtime management of applications. Implementation of essential libraries for P2P communication, VM and container handling and capacity management has been completed. Research of the current state-of-the-art with respect capable monitoring frameworks for Swarms has been done. Analysis and development of TOSCA-based application descriptions that adequately capture monitoring requirements has beecompleted. A first version of the Event Management System (EMS) extensions towards decentralisation capabilities has been implemented. An initial energy optimisation model has been defined.
WP3 completed a SOTA analysis for trust in the Things-to-Cloud continuum, defining trust in the system. It also conducted research and analysis on the Swarmchestrate platform and establishing trust in such an environment. It defined trust, outlined key trust concepts, and explored various trust attributes that contribute towards achieving trustworthiness. The WP also analysed the advantages of using Decentralized Identifiers (DIDs) and Verifiable Credentials (VCs) to establish trustworthy interactions between entities within the Swarmchestrate ecosystem, created an nintial design for the Trust Management System, implemented a preliminary prototype, incorporating direct, context-based, and reputation-based trust, and created an integration plan for embedding the system into the platform. WP3 collected analysed an defined privacy and security requirements of Swarmchestrate services and demonstrators, and started the implementation of a privacy-preserving solution for matchmaking of application requirements and resource capabilities.
In WP4 main system components and their functionalities have been defined, together with a high-level architecture of the Swarmchestrate system. Every subsystem has defined its input and output interfaces as a black-box to support integration. Main features for the demonstrators have been defined and presented in diagrams, requirements lists, scaling rules and their related monitoring metrics. System design for the Swarmchestrate demonstrators applications has been completed, followed by the prototype development of the demonstrator applications. WP4 also designed the model of the Resource Agent for the simulation environment, matching the requirements of the Swarmchestrate deployment mechanism. Finally, the WP Improved the models representing Resource Agents, Applications, and Capacities to simulate the application deployment phase, within the Cloud-to-Edge Continuum.
The implementation of the Swarmchestrate orhestrator, that will represent significant results beyond the state of the art, is currently in progress. Therefore, at this stage of the process, there are no results to report.