Periodic Reporting for period 1 - FLUIDOS (Flexible, scaLable and secUre decentralIzeD Operating System)
Berichtszeitraum: 2022-09-01 bis 2023-08-31
FLUIDOS leverages the enormous, unused processing capacity at the edge, scattered across heterogeneous edge devices that struggle to integrate with each other and to coherently form a seamless computing continuum. By way of a disruptive, open-source paradigm that hinges upon secure protocols for advertisement and discovery, AI-powered resource orchestration and intent-based service integration, FLUIDOS will create a fluid, dynamic, scalable and trustable computing continuum that spans across devices, unifies edge and cloud in an energy-aware fashion, and possibly extends beyond administrative boundaries. FLUIDOS will build upon consolidated Operating Systems and orchestration solutions like Kubernetes, on top of which it will provide a new, enriched layer enacting resource sharing through advertisement/agreement procedures (in the horizontal dimension), and hierarchical aggregation of nodes, inspired by Inter-domain routing in the Internet (in the vertical dimension). Intent-based orchestration will leverage advanced AI Algorithms to optimize costs and energy usage in the continuum, promoting efficient usage of edge resources. A Zero-Trust paradigm will allow FLUIDOS to securely control and access geographically diverse resources, while Trusted Platform Modules will provide strong isolation and guarantee a safe deployment of applications and services. FLUIDOS will pursue the above goals through the creation of an open, collaborative ecosystem, focused on the development of a multi-stakeholder market of edge services and applications, promoting European digital autonomy. The involvement of stakeholders is planned from the outset of the project through pilots and demonstrator in the fields of intelligent energy, agriculture and logistics, which will challenge FLUIDOS capabilities to adapt to different environments and operating conditions, while showcasing its ground-breaking innovation potential.
Current approaches for the computing continuum handle each infrastructure as a multitude of (connected) isolated silos instead of a unique virtual space. This leads to a sub-optimal fragmented view of the overall available resources, preventing the seamless deployment of fully distributed applications. Fragmentation also hinders the potential dynamism in the workload placement, forcing each application to be assigned upfront to a specific infrastructure.
FLUIDOS advocates the opportunity for a novel architectural paradigm, called liquid computing, which builds upon and extends the well-established cloud and edge computing approaches towards an endless computing continuum. The FLUIDOS unique approach to Computing Continuum has three distinctive characteristics that are not matched by alternative proposals: (1) deployment transparency, (2) communication transparency, and (3) resource availability transparency. For instance, FLUIDOS guarantees the above three transparency properties such that:
1. Each micro-service composing a complex application is automatically deployed in the best location of the computing continuum;
2. All communications between micro-services are guaranteed without any modification to the application itself independently from the location of each microservice;
3. A service running in the computing continuum can leverage all the resources belonging to the above virtual domain, independently from the physical cluster that provides that resource.
Focusing mainly on scientific and technical achievements, the major progresses of FLUIDOS towards the objectives during the first 12 months of the project are the following:
● Analyzed possible business models for “fluid” computing with major stakeholders in order to assess long-term the sustainability of the approach (WP1), analyzed current and future market opportunities (WP7); explored the possible advantages of the “fluid” approach in terms of energy consumption (WP6).
● Defined the first version of the FLUIDOS architecture (WP2-6) and released a first PoC implementation (WP2-WP3).
● Defined the first ontology which provides a flexible and extendible way to define what can be advertised between FLUIDOS nodes (WP3, WP4).
● Defined a first version of the REAR (REsource Advertisement and Reservation) protocol (WP2-4; WP5 to be included in Y2), which can advertise, negotiate and reserve resources (services will be added in Y2) between multiple FLUIDOS nodes (a.k.a. Kubernetes clusters). First implementation of the protocol with Kubernetes as a backend.
● Defined a first version of the node orchestrator, which can coordinate the allocation of resources among different FLUIDOS nodes (WP4-3). Explored several directions for scheduling algorithms to be used in FLUIDOS domains, targeting different operating conditions (WP4).
● Demonstrated a first intent-based interface to be used within a FLUIDOS node (WP4).
● Carried out a seminal work on identifying necessary building blocks for creating AI-based algorithms for use within FLUIDOS framework, especially for use cases such as resource allocation, task scheduling, anomaly detection or orchestration (WP4).
● Identified requirements and challenges to build a secure FLUIDOS computing continuum, which can be split in the main following FLUIDOS phases, namely discovery of peering candidates, resource acquisition (including negotiation, reservation, contract signing, and peering), and usage (WP5).
● Investigated and submitted several security topics such as distributed authentication and authorization (potentially integrated within the Gaia-X framework), segregation of resources and restricting communications and interactions (with contributions into the Liqo.io framework), confidential execution of external workloads, trusted execution environments (WP5). Finally, also the protection of the computing continuum (e.g. anomaly detection, cyber deception) have been considered (WP5).
● Initial study (published) on the potential advantage in terms of energy consumption with respect to a fluidified infrastructure, and prototype of a carbon-aware scheduler for distributing workloads across nodes in time and space (WP6).
● Development of a composable disaggregated infrastructure (CDI) paradigm leveraging nearby (fluid) resources, targeting a more effective infrastructure.
● Derived precise requirements for the FLUIDOS use cases (robotic logistics, intelligent power grid, smart viticulture), created high-level description of each use case for non-technical people and open calls applicants, highlighted new possible use cases. Initial implementation of the use cases with the FLUIDOS technology available at Y1 (WP7).
● Participated to several events and joined dissemination communities to present the FLUIDOS approach to the computing continuum; initial dissemination and preparation for the Open Calls (beginning 2024); explored the possibility to join the Eclipse community for FLUIDOS open-source artifacts (WP8).
● Procedure for source code contributions, main workflows, automatic software testing, and the full CI/CD infrastructure has been setup and documented, having in mind both project partners and external contributors (WP9).
No major challenges that could not be properly handled came up in the first year of the project and no critical risks materialised. In addition, all deliverables were submitted, and milestones were reached as planned. Finally, no major deviations were observed regarding the performed activities and consumed resources.
FLUIDOS advocates the opportunity for a novel architectural paradigm, called liquid computing, which builds upon and extends the well-established cloud and edge computing approaches towards an endless computing continuum. The FLUIDOS unique approach to Computing Continuum has three distinctive characteristics that are not matched by alternative proposals: (1) deployment transparency, (2) communication transparency, and (3) resource availability transparency. For instance, FLUIDOS guarantees the above three transparency properties such that:
1. Each micro-service composing a complex application is automatically deployed in the best location of the computing continuum;
2. All communications between micro-services are guaranteed without any modification to the application itself independently from the location of each microservice;
3. A service running in the computing continuum can leverage all the resources belonging to the above virtual domain, independently from the physical cluster that provides that resource.
Focusing mainly on scientific and technical achievements, the major progresses of FLUIDOS towards the objectives during the first 12 months of the project are the following:
● Analyzed possible business models for “fluid” computing with major stakeholders in order to assess long-term the sustainability of the approach (WP1), analyzed current and future market opportunities (WP7); explored the possible advantages of the “fluid” approach in terms of energy consumption (WP6).
● Defined the first version of the FLUIDOS architecture (WP2-6) and released a first PoC implementation (WP2-WP3).
● Defined the first ontology which provides a flexible and extendible way to define what can be advertised between FLUIDOS nodes (WP3, WP4).
● Defined a first version of the REAR (REsource Advertisement and Reservation) protocol (WP2-4; WP5 to be included in Y2), which can advertise, negotiate and reserve resources (services will be added in Y2) between multiple FLUIDOS nodes (a.k.a. Kubernetes clusters). First implementation of the protocol with Kubernetes as a backend.
● Defined a first version of the node orchestrator, which can coordinate the allocation of resources among different FLUIDOS nodes (WP4-3). Explored several directions for scheduling algorithms to be used in FLUIDOS domains, targeting different operating conditions (WP4).
● Demonstrated a first intent-based interface to be used within a FLUIDOS node (WP4).
● Carried out a seminal work on identifying necessary building blocks for creating AI-based algorithms for use within FLUIDOS framework, especially for use cases such as resource allocation, task scheduling, anomaly detection or orchestration (WP4).
● Identified requirements and challenges to build a secure FLUIDOS computing continuum, which can be split in the main following FLUIDOS phases, namely discovery of peering candidates, resource acquisition (including negotiation, reservation, contract signing, and peering), and usage (WP5).
● Investigated and submitted several security topics such as distributed authentication and authorization (potentially integrated within the Gaia-X framework), segregation of resources and restricting communications and interactions (with contributions into the Liqo.io framework), confidential execution of external workloads, trusted execution environments (WP5). Finally, also the protection of the computing continuum (e.g. anomaly detection, cyber deception) have been considered (WP5).
● Initial study (published) on the potential advantage in terms of energy consumption with respect to a fluidified infrastructure, and prototype of a carbon-aware scheduler for distributing workloads across nodes in time and space (WP6).
● Development of a composable disaggregated infrastructure (CDI) paradigm leveraging nearby (fluid) resources, targeting a more effective infrastructure.
● Derived precise requirements for the FLUIDOS use cases (robotic logistics, intelligent power grid, smart viticulture), created high-level description of each use case for non-technical people and open calls applicants, highlighted new possible use cases. Initial implementation of the use cases with the FLUIDOS technology available at Y1 (WP7).
● Participated to several events and joined dissemination communities to present the FLUIDOS approach to the computing continuum; initial dissemination and preparation for the Open Calls (beginning 2024); explored the possibility to join the Eclipse community for FLUIDOS open-source artifacts (WP8).
● Procedure for source code contributions, main workflows, automatic software testing, and the full CI/CD infrastructure has been setup and documented, having in mind both project partners and external contributors (WP9).
No major challenges that could not be properly handled came up in the first year of the project and no critical risks materialised. In addition, all deliverables were submitted, and milestones were reached as planned. Finally, no major deviations were observed regarding the performed activities and consumed resources.