Periodic Reporting for period 1 - OpenCUBE (Open-Source Cloud-Based Services on EPI Systems)
Okres sprawozdawczy: 2023-01-01 do 2024-06-30
The overall goal of the OpenCUBE project is to provide a pre-production reference implementation of a full software stack that enables Cloud Services on a blueprint EPI-based infrastructure atop of European processors. We achieve the goal through four specific objectives as follows.
Objective I: Design and deploy a hardware platform capable of hosting EPI processor and accelerator and factoring into the requirements of representative cloud workloads
Objective II: Design and deliver a full software stack that enables the initialization of cloud servers on the EPI-based platform
Objective III: Enable system-wide efficiency through heterogeneous data centre architecture and eased programmability of heterogeneous resources.
Objective IV: Fully test and validate the deployed stack with a diverse set of applications including cloud-native, HPC, and emerging coupled workflows to run the deployed stack
Objective I: Design and deploy a hardware platform capable of hosting EPI processor and accelerator and factoring into the requirements of representative cloud workloads
Objective II: Design and deliver a full software stack that enables the initialization of cloud servers on the EPI-based platform
Objective III: Enable system-wide efficiency through heterogeneous data centre architecture and eased programmability of heterogeneous resources.
Objective IV: Fully test and validate the deployed stack with a diverse set of applications including cloud-native, HPC, and emerging coupled workflows to run the deployed stack
Objective I: Design and deploy a hardware platform capable of hosting EPI processor and accelerator and factoring into the requirements of representative cloud workloads.
Summary of achievement so far: we have conducted an analysis of application requirements (completed MS3) and designed the prototype hardware platform to support efficient development of the OpenCUBE stack (completed in MS4). The hardware platform is installed and deployed in HPE Grenoble Data Center. The hardware design and components are documented in D2.1. An initial characterization of the prototype platform is conducted and documented in D4.1. We have also completed a storage upgrade on the hardware platform in the first 18 months.
Objective II: Design and deliver a full software stack that enables the initialization of cloud servers on the EPI-based platform.
Summary of achievement so far: we have conducted studies of existing open-source software solutions in the main components of the OpenCUBE stack, including Operating Systems, resource management, scheduling, and tooling, middleware for heterogeneous compute and memory, disaggregated memory, and high-performance networks. We have documented the initial design of these components in deliverables D3.1 and D4.1 and completed MS4. We have performed prototype implementation and carried out validation activities on the pilot platform as completed in milestone MS5 with initial findings summarized in internal documents shared among consortium.
Objective III: Enable system-wide efficiency through heterogeneous data centre architecture and eased programmability of heterogeneous resources.
Summary of achievement so far: we have conducted studies of open-source monitoring solutions at different levels of a system, from hardware counters to node-level and container-level metrics to enable a holistic view and monitoring of power and energy consumption. The design of the monitoring at different levels of the system is documented in D3.1. For efficient usage of heterogeneous resources, we have also designed and developed profiling tools and documented them in D4.1 which have initial implementation completed and functionalities evaluated on the pilot platform.
Objective IV: Fully test and validate the deployed stack with a diverse set of applications including cloud-native, HPC, and emerging coupled workflows to run the deployed stack.
Summary of achievement so far: we have selected a diverse set of pilot applications in WP5 and analyzed their application requirements on hardware and software in the completed MS2. Via system software set up in WP3 and WP4, we have deployed the cloud-native applications on the installed prototype platform to derive their performance baseline for the full test and validation of the OpenCUBE stack to be developed. We continued to port and test the pilot applications on the pilot hardware platform.
Summary of achievement so far: we have conducted an analysis of application requirements (completed MS3) and designed the prototype hardware platform to support efficient development of the OpenCUBE stack (completed in MS4). The hardware platform is installed and deployed in HPE Grenoble Data Center. The hardware design and components are documented in D2.1. An initial characterization of the prototype platform is conducted and documented in D4.1. We have also completed a storage upgrade on the hardware platform in the first 18 months.
Objective II: Design and deliver a full software stack that enables the initialization of cloud servers on the EPI-based platform.
Summary of achievement so far: we have conducted studies of existing open-source software solutions in the main components of the OpenCUBE stack, including Operating Systems, resource management, scheduling, and tooling, middleware for heterogeneous compute and memory, disaggregated memory, and high-performance networks. We have documented the initial design of these components in deliverables D3.1 and D4.1 and completed MS4. We have performed prototype implementation and carried out validation activities on the pilot platform as completed in milestone MS5 with initial findings summarized in internal documents shared among consortium.
Objective III: Enable system-wide efficiency through heterogeneous data centre architecture and eased programmability of heterogeneous resources.
Summary of achievement so far: we have conducted studies of open-source monitoring solutions at different levels of a system, from hardware counters to node-level and container-level metrics to enable a holistic view and monitoring of power and energy consumption. The design of the monitoring at different levels of the system is documented in D3.1. For efficient usage of heterogeneous resources, we have also designed and developed profiling tools and documented them in D4.1 which have initial implementation completed and functionalities evaluated on the pilot platform.
Objective IV: Fully test and validate the deployed stack with a diverse set of applications including cloud-native, HPC, and emerging coupled workflows to run the deployed stack.
Summary of achievement so far: we have selected a diverse set of pilot applications in WP5 and analyzed their application requirements on hardware and software in the completed MS2. Via system software set up in WP3 and WP4, we have deployed the cloud-native applications on the installed prototype platform to derive their performance baseline for the full test and validation of the OpenCUBE stack to be developed. We continued to port and test the pilot applications on the pilot hardware platform.
The OpenCUBE software framework will provide a pathway for a cloud system to be used in Europe by both the public and private sectors beyond the lifetime of the project. The information in the DoA still stays highly relevant. We are continuing to keep track of EPI timeline for the European processor and accelerator delivery timelines and deployment plans. Technologies coming out of OpenCUBE will be in a great position to provide the software stack for the potential customer, application developer, and end users of European cloud providers. Thus, there are very good synergies that we can draw upon between the EPI activities and the OpenCUBE project.
Scientific Impacts
● Enable cloud-native applications as well as hybrid workloads and hpc applications on the deployment of an EPI system cloud infrastructure
● Knowledge, method, and expertise on the use and deployment of Cloud services on European processors
● International expertise and leadership on enabling the computing convergence Cloud and HPC.
OpenCUBE Contributions: We publish open-access publications on methods developed in developing OpenCUBE stack. We deploy cloud applications and workflows on the OpenCUBE pilot system similar to the planned deployment of an EPI system cloud infrastructure
Technological/ Economic Impacts
● Improved collaboration between industry, early users, and academia
● Improve the technology readiness level of open-source software components in delivering Cloud Services on a European processor based data center for future commuting infrastructures
● Promotion of recommendations for the development of the European processor and accelerator.
OpenCUBE Contributions: We continue collaborating closely with the European Processor Initiate (EPI) EUPilot project developing the current and future European processor and coprocessor technology. We have started the design and development of tooling environment , OS, system software, and middleware needed for testing and validating the performance and characteristics of cloud applications on SiPearl Rhea European Processor in a data center environment.
Societal Impacts
● Achieve a greener cloud infrastructure by providing a holistic energy-efficient solution
● Improvement in the exploitation of European processors for cloud serves.
OpenCUBE Contributions: We contribute the design and development of monitoring components in OpenCUBE at different system levels to integrate energy-awareness on the infrastructure
Scientific Impacts
● Enable cloud-native applications as well as hybrid workloads and hpc applications on the deployment of an EPI system cloud infrastructure
● Knowledge, method, and expertise on the use and deployment of Cloud services on European processors
● International expertise and leadership on enabling the computing convergence Cloud and HPC.
OpenCUBE Contributions: We publish open-access publications on methods developed in developing OpenCUBE stack. We deploy cloud applications and workflows on the OpenCUBE pilot system similar to the planned deployment of an EPI system cloud infrastructure
Technological/ Economic Impacts
● Improved collaboration between industry, early users, and academia
● Improve the technology readiness level of open-source software components in delivering Cloud Services on a European processor based data center for future commuting infrastructures
● Promotion of recommendations for the development of the European processor and accelerator.
OpenCUBE Contributions: We continue collaborating closely with the European Processor Initiate (EPI) EUPilot project developing the current and future European processor and coprocessor technology. We have started the design and development of tooling environment , OS, system software, and middleware needed for testing and validating the performance and characteristics of cloud applications on SiPearl Rhea European Processor in a data center environment.
Societal Impacts
● Achieve a greener cloud infrastructure by providing a holistic energy-efficient solution
● Improvement in the exploitation of European processors for cloud serves.
OpenCUBE Contributions: We contribute the design and development of monitoring components in OpenCUBE at different system levels to integrate energy-awareness on the infrastructure