Secure Big Data Processing in Untrusted Clouds

Project Information

SecureCloud

Grant agreement ID: 690111

DOI

10.3030/690111

Project closed

EC signature date 29 October 2015

Start date 1 January 2016

End date 31 December 2018

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 2 285 377,00

EU contribution

€ 1 499 627,00

Coordinated by

TECHNISCHE UNIVERSITAET DRESDEN
Germany

Periodic Reporting for period 3 - SecureCloud (Secure Big Data Processing in Untrusted Clouds)

Reporting period: 2018-01-01 to 2018-12-31

Summary of the context and overall objectives of the project

Security of applications and their data is of immediate concern to almost all organisations that use cloud computing. This is particularly true for organisations that must comply with strict security policies, including those which process personal data and those supporting society’s most critical infrastructures, such as finance, utilities, health care and smart grids.

Critical infrastructure operators have legitimate concerns about the dependability of applications hosted in third-party clouds: cloud providers are already struggling to give strong security guarantees. The lack of adequate dependability, however, is increasingly becoming the primary barrier to the broad adoption of cloud computing, not only in the critical infrastructure domain but also in all domains in which the survival of a company depends on the reliability of the cloud. Hence, the cloud becomes itself a critical infrastructure for which we need to guarantee sufficient dependability such that we can justifiably place our trust in the hosted applications.

The SecureCloud project (https://www.securecloudproject.eu/) aims to remove technical impediments to dependable cloud computing, i.e. SecureCloud will ensure the security of applications and their data.

The provided SecureCloud platform is an ecosystem of cloud facilities characterized by superior security guarantees, providing protection from attacks by privileged users (e.g. the cloud provider or the system administrator) and software (e.g. the hypervisor). Protection relies on new security extensions recently introduced into commercially available off-the-shelf CPUs. The current implementation is based on Intel SGX, but support for additional platforms might become available in the future. SecureCloud is customizable, since it enables developers to build a cloud-based computing environment based on SGX-enabled containers that matches their personal preferences. SecureCloud is modular, because it allows developers to pick and use only the features that they need/want. SecureCloud is flexible, since it can satisfy a wide range of customers-specific requirements including big data processing, secure intra-cloud communication, precise microservice scheduling and reliable data storage. SecureCloud is interoperable, in that its facilities can be seamlessly integrated with best of breed offerings from the Open Source community.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We started our work with describing uses cases related to big data processing in the area of smart grids. From this we derived the requirements on the SecureCloud platform. We designed the SecureCloud platform and implemented the necessary building blocks. We evaluated the feasibility of the SecureCloud platform with the help of proof-of-concept demonstrators.

The architecture of the SecureCloud platform is depicted in Figure 1. The overall purpose of the SecureCloud platform is to ease the process of development and execution of applications in untrusted clouds. Thereby the general paradigm is that SecureCloud applications consist of a set of microservices which work together to perform the necessary application tasks. Each SecureCloud microservice will be encapsulated in a single container. The SecureCloud platform enhances the well-established Docker infrastructure by adding security-related components and extensions. From a security perspective, the foundation of the secure execution of SecureCloud microservices are trusted execution environments (TEEs) which in turn are based on hardware security features offered by current commodity CPUs. Namely, we chose Intel SGX as our example TEE.

The proposed SecureCloud platform serves the following purposes:

• Provide means for the secure execution of microservices (SecureCloud runtime);
• Schedule and orchestrate microservices in an efficient and effective manner (SecureCloud infrastructure services);
• Provide the ability to connect microservices to large-scale big data applications (SecureCloud platform services).

In SecureCloud we support the SCONE runtime as well as the Intel software development kit (SDK) to leverage SGX protection capabilities of enabled processors. While SCONE offers an environment for the seamless compilation of legacy applications to run entirely within SGX enclaves, the Intel SDK provides an application programming interface (API) to partition the application code into trusted and untrusted parts, giving more control to the developer and a reduced code surface.

For the orchestration of our microservices we extended OpenStack and Kubernetes. We implemented components for secure communication, supporting one-to-one and many-to-many communication. We created a secure, distributed key-value store with enhanced dependability features. We implemented several data processing services based on the map/reduce programming model, secure stream processing of data, and an Intel SGX based version of Apache Spark.

We developed several demonstrators from the smart grid domain, e.g. related to collecting and processing data from smart meters for billing purposes; for detecting faults and fraud in the power grid etc. Thereby these applications do not only have security requirements but must also fulfil strict Quality-of-Service guarantees – something which also can be achieved with the help of the SecureCloud platform.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The SecureCloud platform allows the processing of big data in untrusted clouds which was an unsolved challenge in the area of cloud computing. Our approach – compared to other similar approaches – strives for minimising the so-called trusted computing base, i.e. the amount of code and components one has to trust. Our research results manifest in more than 15 papers published at well-known international conferences. An overview of these papers is given at the project website (https://www.securecloudproject.eu/papers/).

Given that on the one hand cloud computing is seen as a very important and promising building block for more efficient and effective IT utilisation and on the other hand security risks related to cloud computing are a major concern which hinders a broader adaption we believe that the results of the SecureCloud project have the potential to accelerate the acceptance and thus the usage of cloud computing solutions by an order of magnitude. This will not only substantially decrease the costs related to data processing but will also allow to implement services – especially for small and medium sized enterprises – which they could not realise beforehand.

SecureCloud architecture

SecureCloud logo