Periodic Reporting for period 1 - NEMO (Next Generation Meta Operating System)
Okres sprawozdawczy: 2022-09-01 do 2024-02-29
NEMO gets established as the game changer of AIoT-Edge-Cloud Continuum by introducing an open source, flexible, adaptable, cybersecure and multi-technology meta-Operating System. To achieve technology maturity and massive adoption, NEMO leverages and interfaces existing systems, technologies and Open Standards, while introducing novel concepts, tools, testing facilities/Living Labs and engagement campaigns to go beyond current state of the art, make breakthrough research and create sustainable innovation.
NEMO will introduce innovations at different layers of the protocol stack, enabling on-device Cybersecure Federated ML/DRL, deliver time-triggered (TSN) multipath ad-hoc/hybrid self-organized and zero-delay failback/self-healing multi-cloud clusters, multi-technology Secure Execution Environment and on-Service Level Objectives meta- Orchestrator, Plugin and Apps Lifecycle Management and Intent Based programming tools. Moreover, NEMO will be “by design” and “by innovation” cybersecure and trusted adopting state of the art mechanisms such as Mutual TLS and Digital Identity Attestation.
NEMO will be validated in 5 most prominent industrial sectors (i.e. Farming, Energy, Mobility/City, Industry 4.0 and Media/XR) and 8 use cases in 5 +1 Living Labs, utilizing more than 30 heterogenous IoT devices and real 5G infrastructure. The impact will not only safeguard EU position in data economy and applications verticals, but lower energy efficiency, reduce pesticides and CO2 footprint.
NEMO will introduce innovations at different layers of the protocol stack, enabling on-device Cybersecure Federated ML/DRL, deliver time-triggered (TSN) multipath ad-hoc/hybrid self-organized and zero-delay failback/self-healing multi-cloud clusters, multi-technology Secure Execution Environment and on-Service Level Objectives meta- Orchestrator, Plugin and Apps Lifecycle Management and Intent Based programming tools. Moreover, NEMO will be “by design” and “by innovation” cybersecure and trusted adopting state of the art mechanisms such as Mutual TLS and Digital Identity Attestation.
NEMO will be validated in 5 most prominent industrial sectors (i.e. Farming, Energy, Mobility/City, Industry 4.0 and Media/XR) and 8 use cases in 5 +1 Living Labs, utilizing more than 30 heterogenous IoT devices and real 5G infrastructure. The impact will not only safeguard EU position in data economy and applications verticals, but lower energy efficiency, reduce pesticides and CO2 footprint.
NEMO has defined its meta-OS meta-architecture which is a useful blueprint for designing similar meta-OS ecosystems.
NEMO CFDRL extends MLOps to a novel paradigm ensuring cybersecure, federated real-time DRL with a balance between efficiency and ML convergence performance, while supporting the complete ML development and delivery lifecycle towards ML as a Service provisioning and meta-OS orchestration decision making.
NEMO meta-Network Cluster Controller (mNCC) supports intent-based networking for service delivery between NEMO nodes, possibly located across administrative domains. mNCC incorporates an overlay mechanism to establish connectivity between NEMO nodes, which is based on an intent-based interface to accommodate service requests (currently in beta version), allowing alignment of application Service Level Objectives (SLOs) with network provider’s operational actions.
The NEMO meta-Orchestrator serves as the meta-control plane, strategically placed atop existing container orchestration clusters such as Kubernetes (K8s). The NEMO meta-Orchestrator has been designed as a set of microservices, addressing, service orchestration across the continuum, resource (cluster) management, observability, intelligent decision making, as well as intent-based service management for integration with other NEMO components.
NEMO encompasses the Secure Execution Environment (SEE), which part of the NEMO Kernel Space and aims to enhance security and isolation of microservices, as well as secure and seamless migration capabilities on top of Kubernetes.
NEMO provides a set of automation tools, which enable full-stack automation for the NEMO meta-OS. NEMO adopts ZeroOps with automated workload delivery in meta-OS addressed by IaC practices of the meta-orchestrator, offering unified operational runtime for containerized workloads with built-in runtime protection, offered by the Plugins and Applications Lifecycle Manager (LCM) and application observability, provided by PPEF.
The Monetization and Consensus-based Accountability (MOCA) component provides a trusted and secure mechanism for any type of user who offers or consumes resources to the NEMO platform, referring to workloads (applications, services, plugins), data and nodes.
NEMO adopts ZeroTrust and cybersecurity by design. NEMO incorporates a set of modules to provide high levels of security and privacy for an effective OS, which will mainly support efficient Identity Management, Access Control Management and Intercommunication Security. The NEMO PRESS & Policy Enforcement Framework (PPEF) aims to safeguard security, privacy, ethics, cost, performance, environmental requirements and associated concerns that are defined for each of the NEMO-hosted services. PPEF handles policies for workloads similarly to Service Level Agreements (SLAs).
With Eclipse Foundation leading impact creation activities, NEMO commits strongly to open source and its community. NEMO code is already available as open source on Eclipse Research Labs’ Gitlab repository (https://gitlab.eclipse.org/eclipse-research-labs/nemo-project)
NEMO CFDRL extends MLOps to a novel paradigm ensuring cybersecure, federated real-time DRL with a balance between efficiency and ML convergence performance, while supporting the complete ML development and delivery lifecycle towards ML as a Service provisioning and meta-OS orchestration decision making.
NEMO meta-Network Cluster Controller (mNCC) supports intent-based networking for service delivery between NEMO nodes, possibly located across administrative domains. mNCC incorporates an overlay mechanism to establish connectivity between NEMO nodes, which is based on an intent-based interface to accommodate service requests (currently in beta version), allowing alignment of application Service Level Objectives (SLOs) with network provider’s operational actions.
The NEMO meta-Orchestrator serves as the meta-control plane, strategically placed atop existing container orchestration clusters such as Kubernetes (K8s). The NEMO meta-Orchestrator has been designed as a set of microservices, addressing, service orchestration across the continuum, resource (cluster) management, observability, intelligent decision making, as well as intent-based service management for integration with other NEMO components.
NEMO encompasses the Secure Execution Environment (SEE), which part of the NEMO Kernel Space and aims to enhance security and isolation of microservices, as well as secure and seamless migration capabilities on top of Kubernetes.
NEMO provides a set of automation tools, which enable full-stack automation for the NEMO meta-OS. NEMO adopts ZeroOps with automated workload delivery in meta-OS addressed by IaC practices of the meta-orchestrator, offering unified operational runtime for containerized workloads with built-in runtime protection, offered by the Plugins and Applications Lifecycle Manager (LCM) and application observability, provided by PPEF.
The Monetization and Consensus-based Accountability (MOCA) component provides a trusted and secure mechanism for any type of user who offers or consumes resources to the NEMO platform, referring to workloads (applications, services, plugins), data and nodes.
NEMO adopts ZeroTrust and cybersecurity by design. NEMO incorporates a set of modules to provide high levels of security and privacy for an effective OS, which will mainly support efficient Identity Management, Access Control Management and Intercommunication Security. The NEMO PRESS & Policy Enforcement Framework (PPEF) aims to safeguard security, privacy, ethics, cost, performance, environmental requirements and associated concerns that are defined for each of the NEMO-hosted services. PPEF handles policies for workloads similarly to Service Level Agreements (SLAs).
With Eclipse Foundation leading impact creation activities, NEMO commits strongly to open source and its community. NEMO code is already available as open source on Eclipse Research Labs’ Gitlab repository (https://gitlab.eclipse.org/eclipse-research-labs/nemo-project)
NEMO, through our partner TID, has already realized standardization activity with 3 contributions to IETF.
The NEMO innovative concept and approach has been published in 2 papers.
NEMO has also applied its Deep Reinforcement Learning (DRL) algorithms for Multisite gaming streaming optimization over virtualized 5G environmentthat can adapt, in a flexible way, to the different conditions that the multimedia workflow could face. The work has been published as a journal publication.
NEMO has designed and implemented FREDY, a differential private federated learning framework that enables knowledge transfer from private data. Particularly, our approach has a teachers–student scheme, in which each teacher model is trained on sensitive, disjoint data in a federated manner, and the student model is trained on the most voted predictions of the teachers on public unlabelled data which are noisy aggregated in order to guarantee the privacy of each teacher’s sensitive data. Our work has been published in a journal publication
Moreover, proof-of-concept implementation is available for Generative Adversarial Networks (GAN) based data poisoning attack and its mitigation in a Federated Learning system.
Besides Federated Learning, attacks are also considered for Decentralized Federated Learning, specifically for Gossip Learning. Our work has been accepted a in the International Conference on Advanced Information Networking and Applications (AINA) 2024:.
The innovations included in NEMO Secure Execution Engine have been contributed to 2 conference papers.
The NEMO innovative concept and approach has been published in 2 papers.
NEMO has also applied its Deep Reinforcement Learning (DRL) algorithms for Multisite gaming streaming optimization over virtualized 5G environmentthat can adapt, in a flexible way, to the different conditions that the multimedia workflow could face. The work has been published as a journal publication.
NEMO has designed and implemented FREDY, a differential private federated learning framework that enables knowledge transfer from private data. Particularly, our approach has a teachers–student scheme, in which each teacher model is trained on sensitive, disjoint data in a federated manner, and the student model is trained on the most voted predictions of the teachers on public unlabelled data which are noisy aggregated in order to guarantee the privacy of each teacher’s sensitive data. Our work has been published in a journal publication
Moreover, proof-of-concept implementation is available for Generative Adversarial Networks (GAN) based data poisoning attack and its mitigation in a Federated Learning system.
Besides Federated Learning, attacks are also considered for Decentralized Federated Learning, specifically for Gossip Learning. Our work has been accepted a in the International Conference on Advanced Information Networking and Applications (AINA) 2024:.
The innovations included in NEMO Secure Execution Engine have been contributed to 2 conference papers.