Periodic Reporting for period 2 - RAINBOW (AN OPEN, TRUSTED FOG COMPUTING PLATFORM FACILITATING THE DEPLOYMENT, ORCHESTRATION AND MANAGEMENT OF SCALABLE, HETEROGENEOUS AND SECURE IOT SERVICES AND CROSS-CLOUD APPS)
Reporting period: 2021-07-01 to 2022-12-31
RAINBOW’s goal was to let developers, service providers and infrastructure do what they do best without having to develop complex services for smart resource provisioning, monitoring and service orchestration. At the same time, developers, and IT solution providers (with emphasis on the SMEs that cannot invest on expensive proprietary modules) could get at their hands a complete solution which can guarantee device and mesh network security, data protection, identity management, anonymity, and resource integrity at all networking layers. The above-mentioned goal has been achieved through the research and technical output of the project’s work, in the form of RAINBOW platform components.
The H2020-funded RAINBOW project developed an open and secured fog computing platform that aims to advance the management of extensible, diverse and safe IoT services and cross-cloud applications. RAINBOW aspires to address the need to timely process the ever-increasing amount of data continuously gathered from heterogeneous IoT devices and appliances and enable fog computing to reach its true potential by providing: Deployment, Security, Orchestration, Network Fabric and Data Management & Analytics services tailored-made to support scalable and secure edge applications. The RAINBOW solution will provide significant benefits for popular cloud platforms, fog middleware, and distributed data management engines, and will extend the open-source ecosystem by pushing intelligence to the network edge while also ensuring security and privacy primitives across the device-fog-cloud-application stack. RAINBOW provides a set of innovative mechanisms and middleware tools for IoT orchestration, data collection, and decentralized analytics that guarantees network security, data protection, identity management, and resource integrity. The key characteristic of the middleware is the embedded intelligence and remote attestation mechanisms for establishing trust and QoS requirements while coping with performance and network uncertainties.
RAINBOW developed a fog orchestration framework on top of popular cloud management stacks by rethinking the current de-centralized computing and network algorithms for rapid resource provisioning, monitoring, and seamless management of IoT services and cross-cloud applications.
The H2020-funded RAINBOW project developed an open and secured fog computing platform that aims to advance the management of extensible, diverse and safe IoT services and cross-cloud applications. RAINBOW aspires to address the need to timely process the ever-increasing amount of data continuously gathered from heterogeneous IoT devices and appliances and enable fog computing to reach its true potential by providing: Deployment, Security, Orchestration, Network Fabric and Data Management & Analytics services tailored-made to support scalable and secure edge applications. The RAINBOW solution will provide significant benefits for popular cloud platforms, fog middleware, and distributed data management engines, and will extend the open-source ecosystem by pushing intelligence to the network edge while also ensuring security and privacy primitives across the device-fog-cloud-application stack. RAINBOW provides a set of innovative mechanisms and middleware tools for IoT orchestration, data collection, and decentralized analytics that guarantees network security, data protection, identity management, and resource integrity. The key characteristic of the middleware is the embedded intelligence and remote attestation mechanisms for establishing trust and QoS requirements while coping with performance and network uncertainties.
RAINBOW developed a fog orchestration framework on top of popular cloud management stacks by rethinking the current de-centralized computing and network algorithms for rapid resource provisioning, monitoring, and seamless management of IoT services and cross-cloud applications.
The main achievements of the project are:
1. RAINBOW Integrated Platform (through three different releases)
2. Deployment of the RAINBOW platform in three different demonstrators both in virtual and physical set ups
3. 28 scientific publications, co-organized 5 scientific workshops, organized 6 webinars and 2 hackathons. Attended and participated in more than 31 events. Liaison with 9 other EU projects and 1 cloud initiative performing cross dissemination activities
4. Strong presence over the project's social media and website with videos, newsletters, brochures and press releases.
5. Standardisation contributions in Open Horizon, Centaurus and Trusted Computing Group. Furthermore, a proposal was submitted to the Eclipse IoT Community about a Secure Admission Control Protocol for Kubernetes Control Plane targeting IoT devices over Mobile Ad-Hoc Networks
1. RAINBOW Integrated Platform (through three different releases)
2. Deployment of the RAINBOW platform in three different demonstrators both in virtual and physical set ups
3. 28 scientific publications, co-organized 5 scientific workshops, organized 6 webinars and 2 hackathons. Attended and participated in more than 31 events. Liaison with 9 other EU projects and 1 cloud initiative performing cross dissemination activities
4. Strong presence over the project's social media and website with videos, newsletters, brochures and press releases.
5. Standardisation contributions in Open Horizon, Centaurus and Trusted Computing Group. Furthermore, a proposal was submitted to the Eclipse IoT Community about a Secure Admission Control Protocol for Kubernetes Control Plane targeting IoT devices over Mobile Ad-Hoc Networks
The RAINBOW framework:
1. Supports affinity-aware and multi-objective offloading, enabling the execution of IoT services partitioned into segments on different and heterogeneous fog and edge devices based on user-defined QoS, energy, network dynamics, and cost optimization constraints;
2. Embraces low-cost approximate and adaptive monitoring techniques to dynamically adjust the processing and data dissemination rate of IoT services, allowing edge devices to reduce energy consumption and ease the pressure on fog networks;
3. Supports the offline operation of IoT services and edge devices, enabling them to maintain execution of scheduled or responsive operations in the local environment despite having lost connection. RAINBOW will enable the distribution of intelligence within overlay mesh networks so that data does not have to leave the network logical boundaries for fog service analytics. This will allow IoT services to maintain offline intelligence to reduce communication overheads and cope with network uncertainties. Addressing these challenges will allow analytic jobs to run low-power, mobile, and geo-distributed infrastructure for the creation of insights on top of raw data produced and stored across the overlay mesh network.
RAINBOW enables IoT service operators to quickly express complex analytics through high-level directives without knowledge of the programming model of the data processing engine. Thus, IoT operators could only focus on business logic, leaving for RAINBOW the burden of how and where data is stored, and dealing with low-level data analysis aspects including heterogeneous fog resources and optimizing data movement.
RAINBOW provides the only complete, open and applicable-by-anyone solution in the market with such a wide set of complex characteristics based on deep research on the tackled domains, contributing in practice to the competence of the whole EU software industry, with additional emphasis to cloud and edge/fog computing domains.
During the project, the conducted research and development activities have progressed the state of the art in different areas. Representative cases follow:
1. The deployment description of IoT services through the drag n’ drop Service Graph Editor with a comprehensive set of modelling abstractions that go beyond simple resource provisioning constraints that SOTA frameworks facilitate today.
2. The (near-optimal) placement of IoT services on provisioned fog resources. These placement algorithms are realized within RAINBOW’s Kubernetes-compliant Smart Orchestrator. Being compliant with Kubernetes (arguably the most popular cloud orchestrator) aids in RAINBOW adoption as the migration effort is minimized.
3. The network administration that supports encrypted IPv6 data dissemination and re-active routing based on the CJDNS protocol deployed on top of Kubernetes Calico.
4. Establishing “trust” and verifying security primitives across the device-fog-cloud stack through RAINBOW’s “zero-conf” bootstrapping protocols with minimum overhead in terms of resources required to provide the trust mechanisms.
5. Pushing “intelligence” to the network “edge” with -in-place- data management through RAINBOW’s self-adaptive monitoring, intelligent storage fabric extending across the network overlay, and fog-aware streaming analytics engine built on top of Apache Storm.
6. To quickly perform large-scale, repeatable, and reproducible experimentation, a scalable fog emulator for geo-distributed IoT applications has been developed and released as open-source.
1. Supports affinity-aware and multi-objective offloading, enabling the execution of IoT services partitioned into segments on different and heterogeneous fog and edge devices based on user-defined QoS, energy, network dynamics, and cost optimization constraints;
2. Embraces low-cost approximate and adaptive monitoring techniques to dynamically adjust the processing and data dissemination rate of IoT services, allowing edge devices to reduce energy consumption and ease the pressure on fog networks;
3. Supports the offline operation of IoT services and edge devices, enabling them to maintain execution of scheduled or responsive operations in the local environment despite having lost connection. RAINBOW will enable the distribution of intelligence within overlay mesh networks so that data does not have to leave the network logical boundaries for fog service analytics. This will allow IoT services to maintain offline intelligence to reduce communication overheads and cope with network uncertainties. Addressing these challenges will allow analytic jobs to run low-power, mobile, and geo-distributed infrastructure for the creation of insights on top of raw data produced and stored across the overlay mesh network.
RAINBOW enables IoT service operators to quickly express complex analytics through high-level directives without knowledge of the programming model of the data processing engine. Thus, IoT operators could only focus on business logic, leaving for RAINBOW the burden of how and where data is stored, and dealing with low-level data analysis aspects including heterogeneous fog resources and optimizing data movement.
RAINBOW provides the only complete, open and applicable-by-anyone solution in the market with such a wide set of complex characteristics based on deep research on the tackled domains, contributing in practice to the competence of the whole EU software industry, with additional emphasis to cloud and edge/fog computing domains.
During the project, the conducted research and development activities have progressed the state of the art in different areas. Representative cases follow:
1. The deployment description of IoT services through the drag n’ drop Service Graph Editor with a comprehensive set of modelling abstractions that go beyond simple resource provisioning constraints that SOTA frameworks facilitate today.
2. The (near-optimal) placement of IoT services on provisioned fog resources. These placement algorithms are realized within RAINBOW’s Kubernetes-compliant Smart Orchestrator. Being compliant with Kubernetes (arguably the most popular cloud orchestrator) aids in RAINBOW adoption as the migration effort is minimized.
3. The network administration that supports encrypted IPv6 data dissemination and re-active routing based on the CJDNS protocol deployed on top of Kubernetes Calico.
4. Establishing “trust” and verifying security primitives across the device-fog-cloud stack through RAINBOW’s “zero-conf” bootstrapping protocols with minimum overhead in terms of resources required to provide the trust mechanisms.
5. Pushing “intelligence” to the network “edge” with -in-place- data management through RAINBOW’s self-adaptive monitoring, intelligent storage fabric extending across the network overlay, and fog-aware streaming analytics engine built on top of Apache Storm.
6. To quickly perform large-scale, repeatable, and reproducible experimentation, a scalable fog emulator for geo-distributed IoT applications has been developed and released as open-source.