Periodic Reporting for period 2 - SSICLOPS (Scalable and Secure Infrastructures for Cloud Operations)
Período documentado: 2016-08-01 hasta 2018-01-31
Over the past decade, IT workloads have increasingly migrated to “cloud” infrastructures, i.e. homogeneous computing fabrics built from commodity servers, interconnected by Ethernet fabrics and supported by NAS and SAN storage backends, and managed by control software such as OpenStack and Eucalyptus. Two types of such cloud infrastructure exists namely, public clouds and private clouds. Public cloud infrastructure is provided by global “hyper-giants” such as Amazon (EC2), Microsoft (Azure) and Google, but also by more regional providers such as major telecom operators. In addition to those cloud service providers, most enterprises and other large organizations prefer to run critical workloads on private cloud infrastructure in their own datacenters.
However, these companies have a difficult time competing based on achievable performance for their clouds using commodity components and open-source systems at the same cost as the large-scale cloud providers with their custom-based solutions (leveraging economies of scale). Moreover, because of their smaller operations, enterprises running their private clouds do not have the resources to scale their operations quickly on demand, unless they reach out to those public cloud providers they are trying to avoid in the first place.
The SSICLOPS project puts these companies into a better position and reduces the performance gap, offering a unique opportunity for European manufacturers and service providers to supply the market with an urgently needed technology.
The overall objective of SSICLOPS is to empower enterprises to create and operate high- performance private cloud infrastructure that allows flexible scaling through federation with other private clouds without compromising their service level and security. The SSICLOPS federation supports the efficient integration of clouds, no matter if they are geographically co- located or distributed, belong to the same or different administrative entities or jurisdictions: in all cases, SSICLOPS delivers maximum performance for inter-cloud communication, enforces legal and security constraints, and minimizes the overall resource consumption. In such a federation, individual enterprises will be able to dynamically scale in/out their private cloud services: because they dynamically offer own spare resources (when available) and take in resources from others when needed. This allows maximizing own infrastructure utilization while minimizing excess capacity needs for each federation member.
However, these companies have a difficult time competing based on achievable performance for their clouds using commodity components and open-source systems at the same cost as the large-scale cloud providers with their custom-based solutions (leveraging economies of scale). Moreover, because of their smaller operations, enterprises running their private clouds do not have the resources to scale their operations quickly on demand, unless they reach out to those public cloud providers they are trying to avoid in the first place.
The SSICLOPS project puts these companies into a better position and reduces the performance gap, offering a unique opportunity for European manufacturers and service providers to supply the market with an urgently needed technology.
The overall objective of SSICLOPS is to empower enterprises to create and operate high- performance private cloud infrastructure that allows flexible scaling through federation with other private clouds without compromising their service level and security. The SSICLOPS federation supports the efficient integration of clouds, no matter if they are geographically co- located or distributed, belong to the same or different administrative entities or jurisdictions: in all cases, SSICLOPS delivers maximum performance for inter-cloud communication, enforces legal and security constraints, and minimizes the overall resource consumption. In such a federation, individual enterprises will be able to dynamically scale in/out their private cloud services: because they dynamically offer own spare resources (when available) and take in resources from others when needed. This allows maximizing own infrastructure utilization while minimizing excess capacity needs for each federation member.
The project studied private cloud infrastructure and the performance instrumentation of cloud hypervisors and operating systems. The project worked towards improving the datacenter fabric latency through making various protocol enhancements, and improvements on the network stack and APIs of the data center operating systems
* The project designed and implemented a framework for federated private clouds. This included the development of dynamic workload allocation and scheduling models between datacenters, and solutions for high-throughput and low-latency communication between datacenters
* The project worked on policy and security solutions for federated datacenters, to preserve privacy of the cloud users and the storage systems in the cloud. A main part of this work was to develop the Compact Privacy Policy Language (CPPL) that allows expressing the policy constraints in federated cloud environment. The project also developed security hardened transport protocols for the cloud.
The solutions developed in the project were validated through four workload models based on real use cases brought in by project partners:
* In-memory databases that span across multiple cloud domains
* Computationally intensive high-energy physics analysis leveraging multiple distributed cloud domains
* Network function virtualization at a large European network operator
* Content delivery and caching of latency-sensitive reputation information of network data objects
* The project designed and implemented a framework for federated private clouds. This included the development of dynamic workload allocation and scheduling models between datacenters, and solutions for high-throughput and low-latency communication between datacenters
* The project worked on policy and security solutions for federated datacenters, to preserve privacy of the cloud users and the storage systems in the cloud. A main part of this work was to develop the Compact Privacy Policy Language (CPPL) that allows expressing the policy constraints in federated cloud environment. The project also developed security hardened transport protocols for the cloud.
The solutions developed in the project were validated through four workload models based on real use cases brought in by project partners:
* In-memory databases that span across multiple cloud domains
* Computationally intensive high-energy physics analysis leveraging multiple distributed cloud domains
* Network function virtualization at a large European network operator
* Content delivery and caching of latency-sensitive reputation information of network data objects
The SSICLOPS project advanced the technical state of the art in multiple areas, as outlined below.
The project developed cloud federation mechanisms and implemented them to build a geographically distributed OpenStack-based testbed that integrates the solutions developed in the project within a single testbed. The testbed applies secure and robust multipath communication based on multipath TCP to connect the multiple cloud domains and load balancing mechanisms that were developed in the project. In addition, the project developed modeling and measurement tools for the cloud by developing a P4 language benchmarking suite for programmable FPGA-based systems, and a rapid prototyping framework for P4.
SSICLOPS developed various enhancements to intra-cloud infrastructure technologies, such as operating system communication stack and APIs to improve performance, and implementing high performance scheduling and filtering mechanisms. SSICLOPS also improved virtualisation techniques to enhance cloud performance, for example, by improving the I/O performance in virtualized systems. Furthermore, the project developed new transport protocols for data centers to enhance the secure communication within a single cloud domain (using the NDP protocol), between multiple cloud domains (leveraging secure multipath communication), and towards the user/client (working on new modern transport protocols, such as QUIC).
SSICLOPS improved the privacy and security framework in cloud systems by developing CPPL, a Compact Privacy Policy Language that allows specifying policies for operating in a federated cloud environment, regarding, for example, privacy and security requirements or performance. We also developed mechanisms for secure processing within clouds.
SSICLOPS applied the developed technologies in four use case scenarios (in-memory databases, high-energy physics computation, network function virtualization and delivery of latency-sensitive information), thereby proving their applicability in real-life use.
The software developed in the project will be available as open source.
SSICLOPS researchers have actively participated in standardisations (IETF, ETSI), including co-chairing relevant working groups, and presented papers in top scientific conferences to make the solutions developed in the project available for the wider community. A notable achievement in this respect was a best paper award in SIGCOMM.
SSICLOPS is a highly technical project in which the socio-economic impact and the wider societal implications of the project are not as evident since the objectives of the project were not directed towards societal based problems. However, the socio-economic impact can be seen in a financial business environment in which the solutions developed within the project will allow greater opportunities for smaller, private-based cloud enterprises to compete against large-scale public cloud providers in the areas of scalability and performance, thereby closing the gap in terms of performance competitiveness and potential reduction of costs and resources used.
The project developed cloud federation mechanisms and implemented them to build a geographically distributed OpenStack-based testbed that integrates the solutions developed in the project within a single testbed. The testbed applies secure and robust multipath communication based on multipath TCP to connect the multiple cloud domains and load balancing mechanisms that were developed in the project. In addition, the project developed modeling and measurement tools for the cloud by developing a P4 language benchmarking suite for programmable FPGA-based systems, and a rapid prototyping framework for P4.
SSICLOPS developed various enhancements to intra-cloud infrastructure technologies, such as operating system communication stack and APIs to improve performance, and implementing high performance scheduling and filtering mechanisms. SSICLOPS also improved virtualisation techniques to enhance cloud performance, for example, by improving the I/O performance in virtualized systems. Furthermore, the project developed new transport protocols for data centers to enhance the secure communication within a single cloud domain (using the NDP protocol), between multiple cloud domains (leveraging secure multipath communication), and towards the user/client (working on new modern transport protocols, such as QUIC).
SSICLOPS improved the privacy and security framework in cloud systems by developing CPPL, a Compact Privacy Policy Language that allows specifying policies for operating in a federated cloud environment, regarding, for example, privacy and security requirements or performance. We also developed mechanisms for secure processing within clouds.
SSICLOPS applied the developed technologies in four use case scenarios (in-memory databases, high-energy physics computation, network function virtualization and delivery of latency-sensitive information), thereby proving their applicability in real-life use.
The software developed in the project will be available as open source.
SSICLOPS researchers have actively participated in standardisations (IETF, ETSI), including co-chairing relevant working groups, and presented papers in top scientific conferences to make the solutions developed in the project available for the wider community. A notable achievement in this respect was a best paper award in SIGCOMM.
SSICLOPS is a highly technical project in which the socio-economic impact and the wider societal implications of the project are not as evident since the objectives of the project were not directed towards societal based problems. However, the socio-economic impact can be seen in a financial business environment in which the solutions developed within the project will allow greater opportunities for smaller, private-based cloud enterprises to compete against large-scale public cloud providers in the areas of scalability and performance, thereby closing the gap in terms of performance competitiveness and potential reduction of costs and resources used.