Skip to main content

Scalable management Of LArge Scale cloud computing environments using enhanced software-defined networking technologies

Periodic Report Summary 1 - SOLAS (Scalable management Of LArge Scale cloud computing environments using enhanced software-defined networking technologies)

SOLAS is a Marie Sklowdowska Curie Industry Academic Pathways and Partnerships project running from October 2013 to September 2017. Working in close cooperation, the consortium partners from academia (Waterford Institute of Technology, KTH — Royal Institute of Technology and ETH Zurich) and industry (EMC and Amadeus SAS) will make significant contributions to the state-of-the-art in the increasingly important area of resource management for cloud computing. In performing and evaluating this research programme, the project will train a cohort of academic and industrial researchers that can continue to make significant contributions beyond the project lifetime.

The SOLAS research focus is on resource management for cloud computing, which is widely acknowledged as a hard problem due to: the scale of modern data centres; the heterogeneity of resource types and their interdependencies; the variability and unpredictability of the load; and to the range of objectives of the different actors in a cloud ecosystem. We argue that developing resource management techniques and processes that overcome these issues requires significant foundational and applied research addressing the following challenges:

-- Achieving predictable performance: from a data centre operator’s perspective efficient use of resources implies statistical multiplexing of resources across customers and applications. However, this means that the performance of resources allocated at any given time are less predictable than those of a traditional, dedicated infrastructure;
-- Achieving global manageability: controlling a cloud environment to achieve a set of global, system-wide objectives (which may change over time), requires coordinated management of different resource types and the ability to understand and exploit trade-offs between the usage of different resource types;
-- Engineering scalable resource management systems: to efficiently use cloud infrastructure resource management systems must scale across several dimensions, so that static portioning into resource pools and consequent sub-optimal usage patterns can be avoided in favour of a small number of flat resource pools;
-- Achieving consistent and efficient data management: cloud hosted applications can be viewed as distributed information systems that store and replicate sometimes huge volumes of data across a number of servers, and which must thus be cognisant of the limitations imposed by the potentially significant latency and bandwidth associated with transferring data between servers.

The SOLAS approach is founded on the enhancement of nascent Software-Defined Networking (SDN) technologies and their application in data centre environments. Software Defined Networking is a technique that usually refers to and relies on the decoupling of the control plane functionality and the data/forwarding plane functionality in a network. Through SDN it is possible to provide a coherent view of available network resources and to make those resources highly programmable and thus better and more swiftly adapt to demands as they may change by the applications and services making use of the network for their connectivity needs. Application of SDN in data centre environments is attractive as it offers the potential to significantly improve the monitoring and control of networking resources, which today are extremely difficult to efficiently allocate in data centres and thus are the source of many performance issues.

In light of the above, the SOLAS research programme addresses the long-term research question of how we design and engineer management systems for cloud computing data centres that can provide predictable performance for cloud-hosted applications, can facilitate flexible global manageability and that are scalable to very large, heterogeneous resource pools supporting large numbers of applications? To answer this question we are pursuing the following research objectives:

1. To design and evaluate distributed, self-organising algorithms for ongoing monitoring and real-time searching of system operational data and to investigate their enhancement to provide distributed analytics processes that mine this data to provide insights that help improve fault, performance and security management processes;
2. To design and evaluate scalable, decentralised processes for joint allocation of data centre compute, networking and storage resources satisfying management objectives that can be changed at runtime;
3. To specify an enhanced software-defined networking framework that facilitates scalable application controlled injection of networking functionality into the data plane within data centre environments;
4. To develop novel architectures and data management techniques that exploit software-defined networking to efficiently and seamlessly utilise all system resources (in particular networking resources).

During the initial phase of SOLAS partners made good progress towards fulfilling these research objectives, with a number of papers having been published and submitted by publication based on results generated using research prototypes and real data sets used during secondments between the academic and industry partners. An example of this progress include the development of the MAPLE framework for network-aware virtual machine placement based on empirical effective bandwidth estimates that was developed by Waterford Institute of Technology and that is being enhanced with SDN functionality in cooperation with EMC. Another example is the collaboration between Waterford Institute of Technology and Amadeus SAS which focussed on development of a novel globally measurement for large scale enterprise data centres and on how SDN functionality could be used to meet latency targets for complex Service-Oriented Architecture transactions in enterprise data centres.

For more information on SOLAS please visit the project website at and/or contact the project coordinator:

Dr Brendan Jennings
SOLAS Project Coordinator
Telecommunications Software & Systems Group
Waterford Institute of Technology