Periodic Reporting for period 2 - COLA (Cloud Orchestration at the Level of Application)
Okres sprawozdawczy: 2018-07-01 do 2019-09-30
Cloud computing has successfully and steadily addressed issues of how to run applications on complex distributed computing infrastructures. On-demand access to cloud resources in a flexible and elastic way could result in significant cost savings due to more efficient and convenient resource utilization. However, the efficient and dynamic utilization of Cloud is not trivial. The take up of cloud computing in some application areas is still relatively low due to limited application-level flexibility and shortages in cloud specific skills. Public sector organizations and SME are increasingly considering using the Cloud in their everyday activities but they still face difficulties of both economic and technical nature. To enable the execution of applications in the Cloud in a cost effective, flexible and secure way, applications must be deployed, executed and removed through a framework that hides cloud specific details from users. COLA addressed the following challenges:
- describing the structure of containerized/virtualized applications and their behaviour to control their lifecycle in a cloud agnostic way,
- supporting deployment and run-time orchestration and optimisation of such applications taking various QoS parameters into account,
- creating and running near production level applications in the Cloud.
COLA developed the MiCADO framework to process application descriptions, to deploy and execute them in the Cloud. COLA elaborated the TOSCA based Application Description Template (ADT) to specify applications's structure and their behaviour. ADT enables application descriptions based on two-level topology (container- and Virtual Machine-level) and adding security and scaling policies. ADTs are forwarded to the MiCADO Submitter that parses and validates ADTs and forwards them to adaptors, such as container adaptor, cloud orchestration adaptor and policy keeper adaptor. The orchestrators create and launch virtual machines and containers specified in ADTs. The MiCADO Policy Keeper scales up and down virtual machines and containers using cloud (e.g. Occopus or Terraform) and container orchestrator (e.g. Docker Swarm or Kubernetes) based on the information gathered by the Prometheus monitoring, recommendations provided by the MiCADO Optimizer and TOSCA policies specified in ADTs. The MiCADO Security Policy Manager handles security policies given in ADTs through security enablers. The MiCADO framework can be deployed as Ansible playbook to make deployment effortless as much as possible. To support application developers and end users MiCADO was extended with a dashboard including Docker Visualizer, (replaced by Kubernetes dashboard later), Prometheus. COLA elaborated 3 near production level applications and 26 proof of concepts demonstrators.
- describing the structure of containerized/virtualized applications and their behaviour to control their lifecycle in a cloud agnostic way,
- supporting deployment and run-time orchestration and optimisation of such applications taking various QoS parameters into account,
- creating and running near production level applications in the Cloud.
COLA developed the MiCADO framework to process application descriptions, to deploy and execute them in the Cloud. COLA elaborated the TOSCA based Application Description Template (ADT) to specify applications's structure and their behaviour. ADT enables application descriptions based on two-level topology (container- and Virtual Machine-level) and adding security and scaling policies. ADTs are forwarded to the MiCADO Submitter that parses and validates ADTs and forwards them to adaptors, such as container adaptor, cloud orchestration adaptor and policy keeper adaptor. The orchestrators create and launch virtual machines and containers specified in ADTs. The MiCADO Policy Keeper scales up and down virtual machines and containers using cloud (e.g. Occopus or Terraform) and container orchestrator (e.g. Docker Swarm or Kubernetes) based on the information gathered by the Prometheus monitoring, recommendations provided by the MiCADO Optimizer and TOSCA policies specified in ADTs. The MiCADO Security Policy Manager handles security policies given in ADTs through security enablers. The MiCADO framework can be deployed as Ansible playbook to make deployment effortless as much as possible. To support application developers and end users MiCADO was extended with a dashboard including Docker Visualizer, (replaced by Kubernetes dashboard later), Prometheus. COLA elaborated 3 near production level applications and 26 proof of concepts demonstrators.
COLA developed a generic framework called MiCADO (Microservices-based Cloud Application-level Dynamic Orchestrator) to enable public sector organisations and SMEs to deploy and run applications in the Cloud in order to increase European technological capacity. The framework will increase the number of prospective cloud users in public service sector and SMEs. Running applications in the Cloud will improve services provided to European citizens and companies. The MiCADO framework enables users to deploy and run applications in cost effective, flexible, seamless and secure way in the Cloud. The framework provides the missing link between existing non-cloud aware applications and the dynamic capabilities of IaaS Clouds on demand. MiCADO is technology agnostic because its services are not restricted to particular technologies and can be implemented using different technologies. It is generic as MiCADO can be connected to multiple cloud middleware and generic cloud access layers to avoid dependence on one particular cloud technology. It is based on existing low-level cloud container technologies (e.g. Docker, Swarm, etc.), orchestration solutions (e.g. Occopus), and existing standards (e.g. TOSCA). MiCADO provides access to both academic clouds (OpenNebula and OpenStack) and commercial clouds (Amazon and CloudSigma). The framework is cost-effective as it supports flexible and optimal deployment and run-time orchestration of applications in the Cloud. The MiCADO security architecture delivers the required security services. The key component of this architecture is the Security Policy Manager that handles security enablers to deliver the required security services to secure the deployment and execution environment. COLA developed the concept of Application Description Template (ADT), based on the TOSCA language specification, to support a two-level (container and virtual machine-level) topology, and extended the TOSCA policy specification adding several security and scaling-related policies. In the latest release of the MiCADO Framework the MiCADO Submitter parses, validates and forwards the TOSCA descriptions through adaptors to container orchestrator (Docker Swarm), cloud orchestrator (Occopus) and policy manager (MiCADO Policy Keeper) components. The Policy Keeper microservice is able to scale up and down containers and virtual machines using Docker Swarm and Occopus based on the information gathered by the Prometheus-based monitoring component, and based execution and performance parameters given in TOSCA policies. COLA already demonstrated the usability of MiCADO on several large scale demonstrators. .
MiCADO supports the optimised deployment and execution of applications in IaaS Clouds. The framework is generic, pluggable, technology agnostic and open source. When compared to existing solutions, it enhanced the state of the art in cloud technologies by providing scalability based on highly flexible and extendable user-defined policies. It accelerates the take up of cloud computing by addressing application- and infrastructure-level complexity. The framework covers the entire application lifecycle hiding most of the cloud specific details using TOSCA-based Application Description Templates. The MiCADO concept manages applications as black boxes as the ADT allows description, deployment and execution of applications in the Cloud hiding most of the cloud specific technical details. Application Developers can specify applications and publish their description in a repository. They can re-use existing templates considering the application requirements. Application Developers and End Users can download and submit these descriptions to the framework to deploy and run the applications in the Cloud. COLA also develops a Security Enforcer to guarantee specific security requirements of applications. The TOSCA based application description adds flexibility to security measures specifying them as security policies.
COLA already developed prototypes of three large scale demonstrators (Audience Finder application, evacuation planning, and improving services for citizens), and one proof of concept application (fast data transfer among different type of distributed storages). COLA partners identified further 11 applications that can be considered to be deployed and executed through MiCADO in the Cloud. These applications can demonstrate how to use MiCADO in different application domains focusing on the public sector and SMEs. The “selling point” of this framework is that Application Developers can create cloud-enabled applications from existing applications with minimum efforts and make them available End Users.
COLA already developed prototypes of three large scale demonstrators (Audience Finder application, evacuation planning, and improving services for citizens), and one proof of concept application (fast data transfer among different type of distributed storages). COLA partners identified further 11 applications that can be considered to be deployed and executed through MiCADO in the Cloud. These applications can demonstrate how to use MiCADO in different application domains focusing on the public sector and SMEs. The “selling point” of this framework is that Application Developers can create cloud-enabled applications from existing applications with minimum efforts and make them available End Users.