HIERARCHICAL APPROACH FOR GREEN WORKLOAD MANAGEMENT IN HETEROGENEOUS AND INTERCONNECTED DATA CENTERS

Project Information

EcoMultiCloud

Grant agreement ID: 712363

DOI

10.3030/712363

Project closed

EC signature date 19 January 2016

Start date 1 December 2015

End date 30 November 2017

Funded under

• SOCIETAL CHALLENGES - Secure, clean and efficient energy

Total cost

€ 1 067 915,00

EU contribution

€ 747 208,88

Coordinated by

ECO4CLOUD SRL
Italy

Periodic Reporting for period 2 - EcoMultiCloud (HIERARCHICAL APPROACH FOR GREEN WORKLOAD MANAGEMENT IN HETEROGENEOUS AND INTERCONNECTED DATA CENTERS)

Reporting period: 2016-12-01 to 2017-11-30

Summary of the context and overall objectives of the project

The explosion of digital content, big data, e-commerce, and Internet traffic is making DCs one of the fastest-growing consumers of electricity in developed countries: nowadays, DCs around the world consume about 30 billion watts of power annually. An efficient utilization of resources is essential to reduce operational costs, energy consumption, carbon emissions and to ensure that the Quality of Service experienced by users is adequate to the Service Level Agreements. The adoption of interconnected DCs is rapidly increasing: multinational enterprises, financial institutions, telco operators and major cloud providers are deploying distributed DCs to interconnect both proprietary and third-parties heterogeneous DCs (“Inter-Cloud” scenario). Inter-DC VM migration is a more novel research topic; while opportunities opened by long distance migrations are huge, involved issues are extremely complex: among them, the benefits-to-drawbacks convenience for the workload migrations, the selection of which site from/to migrate, the selection of the workload portion to be migrated, the determination of the reassignment of the migrating workload into the target site. To efficiently address these problems, it is necessary to decentralize part of the intelligence and distribute the decision points, while exploiting the centralized architecture and functionalities offered by virtualization infrastructures in single DCs. This naturally leads to a hierarchical infrastructure, in which single DCs manage the local workload autonomously but communicate with each other to route and migrate VMs among them. In this scenario, Eco4Cloud developed a hierarchical framework for the efficient distribution of the workload on a multi-site scenario allowing for an integrated, homogeneous management of heterogeneous platforms at the same time preserving the autonomy of single sites. EcoMultiCloud gives DC administrators the opportunity to achieve both technical and business goals like reduction of power consumption/carbon emissions, reduction of energy costs, Quality of Service Management, Compliance with SLA and Reduced Inter-DC communications. The following objectives have been achieved during the project:
#1: Complete development and integration of EcoMultiCloud modules and infrastructure;
#2: Assessment of EcoMultiCloud effectiveness and final validation in a real case pilot;
#3: Successful final business innovation plan and pre-commercialization activities.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Eco4Cloud performed the analysis of the different policies for workload allocation/redistribution and the integration of the developed modules to verify their correct functioning and proper inter-operation. Key performance parameters related to costs, load balancing, energy consumption, etc. were assessed and tests conducted using a Java simulator validated respect to real data available at Telecom Italia DCs. The use of an iterative and incremental development design approach was used to perform the modules integration. The tasks led to the identification of the steps needed by the user to interact and operate on the system and to verify its state; particular care was given to the supervision of communication issues between modules.
Starting from the data available at Eco4Cloud regarding the TIM infrastructure, HW resource metrics like CPU, RAM and Disk usage were listed and taken into account, and performance indexes (CPU ready time, ballooned memory and active memory) investigated. Migrations among the TIM data centers were detailed, and collected data was processed in order to demonstrate the impact of migrations in the whole system. The obtained results showed that the quality of the service perceived by the user is high, thus confirming the validity of our approach regarding remote migrations.
The algorithms were customized according to the features of each selected DC owned by TIM (located in Rozzano, DC1, Pomezia, DC2 and Bari, DC3). Collection of the results on the testbed performance involved the parameters on carbon emissions for the three DCs, the energy cost, the consumed energy, the distribution of the workload among the DCs and the number of migrations performed. The alpha version of the EcoMultiCloud software was successfully released.
The final pilot tests of the product were planned and conducted concretely. The three DCs, initially considered as equal in architecture, number and hosts capacity, were characterized with their real features, the assignment function driving the workload distribution among the different DCs was customized, and the pilot tests were performed, first in a passive mode and then in an active mode; performance data was collected for the assessment of the tests results. The beta version of the software, updated and improved with the support of TIM staff, was released. Constraints on workload mobility were taken into account in the final version of the SW, the algorithm further refined and the GUI appearance and usability improved. Finally, resources have been devoted to the implementation of commercial and communication activities; a dedicated dissemination plan was implemented supporting the diffusion of the EcoMultiCloud project results; a final business plan was prepared, and financial forecasts were updated with the most recent estimates on production costs and product prices.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

In order to achieve a successful marketing strategy, we have investigated the impact of our technology on the virtualized DC design operation with a focus on the technological barriers, including:
- Integration with other DC management tools - EcoMultiCloud architecture can be easily integrated with facility-side DC Infrastructure Management tools.
- Availability on multiple virtualization platform – EcoMultiCloud works on top of any virtualization platform, ideal for hybrid environments;
- The amount of inter-DC bandwidth available could be a limiting factor – The adaptive and self-organizing nature of our algorithm allows to handle the problem of the amount of inter-DC bandwidth available.
- The challenges posed by increased workload portability among distributed DCs- our solution envisions that all DCMs in a group of EcoMultiCloud-enabled facilities will communicate with each other.
The current year is going to be the first profitable fiscal year for Eco4Cloud. The profit and loss account forecast of the company estimates a sales revenue growth for the company of 200% year on year, thanks to the features introduced in the Eco4Cloud products suite after completing the EcoMultiCloud project, financed by the H2020 SME INST. Using EcoMultiCloud in the private cloud enables companies to achieve their SLAs at the lowest cost, mainly through workload migration and applications stacking. In a hybrid cloud environment, EcoMultiCloud helps with dynamic placement decisions according to policy, removing people from the provisioning/brokering process. It also enforces policies that reduce sprawl and wasted costs, increases policy compliance, and reduces risks from accidental or malicious activities. Dynamic optimization technology for the public cloud enables better utilization of services at lower costs. The feature of multiple clouds management, provided by the EcoMultiCloud project, allows the company to make its products suite more appealing for companies with large IT infrastructures geographically dispersed, and therefore will allow to increase sales, that will exceed 1M€ in 2019.

ECOMULTICLOUD Hierarchical Architecture