Skip to main content

Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications

Deliverables

Report on RECAP Concertation V.1

This deliverable will provide details about the Concertation Plan and a progress report on concertation activities and impact against planned metrics in the first 12 months.

Final infrastructure orchestration and optimisation

The final and updated version of the D8.2 deliverable. It will capture the developed orchestration and optimisationmethods as well as the complete simulation framework.

Final Requirements and Validation Plan

Based on the results and insights gained from the first integrated system, this deliverable re-visits and finalises the requirements and puts up a plan of how the RECAP shall be validated.

Initial infrastructure modelling and resource mapping

Initial models for the representation of infrastructure landscape, dependencies and configuration of virtual and physical infrastructure resources. This deliverable will capture also preliminary load translation models and methods as the outcome of the tasks 8.2 and 8.3.

Final data acquisition and analytics models

This deliverable will provide the final data acquisition and data analysis mechanisms, refined and tuned based on experiences from the initial versions and the developments of WP6, WP7 and WP8

Initial data acquisition and analytics models

This deliverable will provide initial data acquisition mechanisms for offline (trace based) data analysis and online (monitoring based) data acquisition. It will deal with preliminary analytics models providing fundamental data filtering, aggregation, pre-processing and analysis capabilities

Initial infrastructure orchestration and optimisation

This deliverable will capture the initial application and resource orchestration approaches and utility functions. In addition, it will document improved placement and re-placement optimisationmethods, and holistic configuration optimisationmethods. Statistical and machine learning methods that (semi-)automatically can recognize and classify patterns in data centre configurations, and the contributions of application structures and components to these. Simulation capabilities will be delivered for evaluation and experimentation with different types of applications and optimisationmethods.

Initial Requirements

This document details the constraints, wishes, and requirements for each of the use cases and testbed operators. This not only holds for functional requirements, but also captures quality of service aspects and sketches the technical framework such as APIs that have to be used.

Initial System Architecture and Integration

This deliverable documents the system architecture and the way the software artefacts from the various technical work packages will beassembled in order to getit a working, integrated system(D4.3)

Intermediate simulation platform: Applications and workload mapping

This deliverable will demonstrate how the models have been mapped into the simulation platform. In addition, the intermediate validation of the models against real measurements will be shown in this deliverable. The simulation capabilities will be delivered for evaluation and experimentation with different types of applications and optimisationmethods.

Final Quality-of-service metrics and models

This deliverable accounts for the outcomes of T7.1 and T7.2.

Final Workload, Load Propagation, and Application Models

Advancedworkload and application models that a fine granularity capture the composition and statistical properties of request-based workloads for data centre applications, and provide stable predictive capabilities including, e.g., proactive auto-scaling techniques, burst and anomaly detection techniques, and provisioning of simulation models that allow (re-)construction of artificial traces that retain the statistical properties of the original traces. Load propagation models that build on the preliminary load models and provide more advanced application elasticity and remediation capabilities such as probabilistic estimations for alternative load paths and component health-monitoring based recovery mechanisms that can be used in deployment optimisationand self-tuning of applications.

Validation Report

This deliverable documents the results of the validation and aligns the requirements with the actual capabilities of the platform.

Report on RECAP Concertation V.3

This deliverable will include a review of all concertation activities undertaken during the project.

Initial workload, Load propagation, and Application Models

Initial workload and application models (including workload decomposition and characterization methods) that capture workload characteristics at a fine-grained scale and model the propagation of request patterns through application structures (including the application response to workloads) simultaneously. Preliminary versions of simulation tools facilitation basic experimentation with workloads, including, e.g., replaying and visualizing load patterns

Report on RECAP Concertation V.2

This follow-up deliverable will provide updates on targets, detailed figures and progress towardsimplementing the concertation plan.

Final System Architecture and Integration

The second iteration of the architecture and integration deliverable

Final infrastructure modelling and resource mapping

Final resource management models that extend the capabilities of the initial versions and include models for more accurate prediction of resource requirements and mapping of the high level workload behaviour to low level resource landscape.

Initial Quality-of-service metrics and models

This deliverable will describe quality-of-service metrics and models for the characterization of quality-of-service provisioning the next-generation cloud architectures. The content of this deliverable will be fed to T7.1 and T7.2, in order to be integrated into the simulation framework.

Initial Testbed Configuration

This deliverable documents the installation and configuration of the testbeds provided by UULM.In particular, it aligns the technical set-up with the requirements defined in D3.1. Moreover, the document serves as a manual documenting how consortium partner can make use of it.

RECAP initial integrated prototype

This deliverable integrates the software artefacts from Deliverables D5.1, D6.1, D7.2, D8.1, and D8.2 in a distributed software platform following the architecture and paradigms described in D4.2. The outcome is a basic building block for the update testbeds

Improved Testbed Configuration

This document updates D4.1 taking into account the experiences gained from the first integration and the changes made to address potentially changed and refined requirements in D3.2.

Methods and framework for data visualization

This deliverable will define methods, frameworks, and web-accessible APIs to create and customize data visualization, enabling the inspection of application structures, deployments, mappings to infrastructure resources, and load propagation patterns.

System Demonstrator

This deliverable documents the realisation of the demonstrators.

Final RECAP simulation platform

The final capabilities of the simulation framework will be presented into this deliverable, including the parallelisation process, the simulation simplification, and their designs.

Initial simulation platform

This deliverable will capture the simulation platform and its capabilities. Main designs and initial tests of the models will be delivered. The deliverable will capture also the process to start the simulation processes automatically.

RECAP Project Website

This deliverable will provide the successful deployment of the project website.

Searching for OpenAIRE data...

Publications

Towards understanding HPC users and systems: A NERSC case study

Author(s): Gonzalo P. Rodrigo, P.-O. Östberg, Erik Elmroth, Katie Antypas, Richard Gerber, Lavanya Ramakrishnan
Published in: Journal of Parallel and Distributed Computing, Issue 111, 2018, Page(s) 206-221, ISSN 0743-7315
DOI: 10.1016/j.jpdc.2017.09.002

Power-performance tradeoffs in data center servers: DVFS, CPU pinning, horizontal, and vertical scaling

Author(s): Jakub Krzywda, Ahmed Ali-Eldin, Trevor E. Carlson, Per-Olov Östberg, Erik Elmroth
Published in: Future Generation Computer Systems, Issue 81, 2018, Page(s) 114-128, ISSN 0167-739X
DOI: 10.1016/j.future.2017.10.044

Analyzing the availability and performance of an e-health system integrated with edge, fog and cloud infrastructures

Author(s): Guto Leoni Santos, Patricia T. Endo, Matheus F. F. S. Lisboa Tigre, Leylane Ferreira, Djamel Sadok, Judith Kelner and Theo Lynn
Published in: Journal of Cloud Computing: Advances, Systems and Applications, 2018, ISSN 2192-113X

Simulating large vCDN networks: A parallel approach

Author(s): Christos K. Filelis-Papadopoulos, Konstantinos M. Giannoutakis, George A. Gravvanis, Patricia Takako Endo, Dimitrios Tzovaras, Sergej Svorobej, Theo Lynn
Published in: Simulation Modelling Practice and Theory, Issue 92, 2019, Page(s) 100-114, ISSN 1569-190X
DOI: 10.1016/j.simpat.2019.01.001

Towards simulation and optimization of cache placement on large virtual content distribution networks

Author(s): Christos K. Filelis-Papadopoulos, Patricia Takako Endo, Malika Bendechache, Sergej Svorobej, Konstantinos M. Giannoutakis, George A. Gravvanis, Dimitrios Tzovaras, James Byrne, Theo Lynn
Published in: Journal of Computational Science, Issue 39, 2020, Page(s) 101052, ISSN 1877-7503
DOI: 10.1016/j.jocs.2019.101052

Self-Service Cybersecurity Monitoring as Enabler for DevSecOps

Author(s): Jessica Diaz, Jorge E. Perez, Miguel A. Lopez-Pena, Gabriel A. Mena, Agustin Yague
Published in: IEEE Access, Issue 7, 2019, Page(s) 100283-100295, ISSN 2169-3536
DOI: 10.1109/access.2019.2930000

Machine Learning Methods for Reliable Resource Provisioning in Edge-Cloud Computing

Author(s): Thang Le Duc, Rafael García Leiva, Paolo Casari, Per-Olov Östberg
Published in: ACM Computing Surveys, Issue 52/5, 2019, Page(s) 1-39, ISSN 0360-0300
DOI: 10.1145/3341145

A Novel Hyperparameter-Free Approach to Decision Tree Construction That Avoids Overfitting by Design

Author(s): Rafael Garcia Leiva, Antonio Fernandez Anta, Vincenzo Mancuso, Paolo Casari
Published in: IEEE Access, Issue 7, 2019, Page(s) 99978-99987, ISSN 2169-3536
DOI: 10.1109/access.2019.2930235

Simulating Fog and Edge Computing Scenarios: An Overview and Research Challenges

Author(s): Sergej Svorobej, Patricia Takako Endo, Malika Bendechache, Christos Filelis-Papadopoulos, Konstantinos Giannoutakis, George Gravvanis, Dimitrios Tzovaras, James Byrne, Theo Lynn
Published in: Future Internet, Issue 11/3, 2019, Page(s) 55, ISSN 1999-5903
DOI: 10.3390/fi11030055

Continuous Anything for Distributed Research Projects

Author(s): Volpert, Simon; Griesinger, Frank; Domaschka, Jörg
Published in: Dependability Engineering, Issue 1, 2018, Page(s) 23-46

The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS

Author(s): Daniel Seybold, Christopher B. Hauser, Georg Eisenhart, Simon Volpert, Jörg Domaschka
Published in: Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Turin, Italy, August 27-28, 2018, Revised Selected Papers, Issue 11339, 2019, Page(s) 93-105
DOI: 10.1007/978-3-030-10549-5_8

Technology, Science, and Culture: A Global Vision

Author(s): Sergio Picazo-Vela, Luis Ricardo Hernández
Published in: Technology, Science and Culture - A Global Vision, 2019
DOI: 10.5772/intechopen.83691

Done Yet? A Critical Introspective of the Cloud Management Toolbox

Author(s): Leznik, Mark; Volpert, Simon; Griesinger, Frank; Seybold, Daniel and Domaschka, Jörg
Published in: 24th IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), 2018

Modeling the availability of an e-health system integrated with edge, fog and cloud infrastructures

Author(s): Matheus Felipe Ferreira da Silva Lisboa Tigre, Guto Leoni Santos, Theo Lynn, Djamel Sadok, Judith Kelner, and Patricia Takako Endo
Published in: IEEE Symposium on Computers and Communications (IEEE ISCC 2018), 2018

"ATMoN: Adapting the ""Temporality"" in Large-Scale Dynamic Networks"

Author(s): Demetris Trihinas, Luis F. Chiroque, George Pallis, Antonio Fernandez Anta, Marios D. Dikaiakos
Published in: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), 2018, Page(s) 400-410
DOI: 10.1109/ICDCS.2018.00047

SAT-IoT: An Architectural Model for a High-Performance Fog/Edge/Cloud IoT Platform

Author(s): Miguel Angel Lopez Pena, Isabel Munoz Fernandez
Published in: 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), Issue 15-18 April 2019, 2019, Page(s) 633-638
DOI: 10.1109/wf-iot.2019.8767282

Adaptive Resource Provisioning based on Application State

Author(s): Constantine Ayimba, Paolo Casari, Vincenzo Mancuso
Published in: 2019 International Conference on Computing, Networking and Communications (ICNC), 2019, Page(s) 663-668
DOI: 10.1109/ICCNC.2019.8685605

Mowgli - Finding Your Way in the DBMS Jungle

Author(s): Daniel Seybold, Moritz Keppler, Daniel Gründler, Jörg Domaschka
Published in: Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering - ICPE '19, 2019, Page(s) 321-332
DOI: 10.1145/3297663.3310303

Towards Understanding the Performance of Distributed DatabaseManagement Systems in Volatile Environments

Author(s): Jörg Domaschka, Daniel Seybold
Published in: Symposium on Software Performance 2019, Issue 10, 2019

Kaa: Evaluating Elasticity of Cloud-hosted DBMS

Author(s): Daniel Seybold, Simon Volpert, Stefan Wesner, Andre Bauer, Nikolas Herbst, Jörg Domaschka
Published in: International Conference on Cloud Computing Technology and Science, Issue 11, 2019

Unified Container Environments for Scientific Cluster Scenarios

Author(s): Benjamin Schanzel, Mark Leznik, Simon Volpert, Jörg Domaschka, Stefan Wesner
Published in: bwHPC Symposium, Issue 5, 2019

Analyzing Resource Distribution over a Real-World Large-Scale Virtual Content Infrastructure

Author(s): Patricia Takako Endo; Radhika Loomba; Ruth Quinn; Christos Filelis-Papadopoulos; Konstantinos Giannoutakis; George A. Gravvanis; Dimitrios Tzovaras; Peter Willis; Sergej Svorobej, James Byrne and Theo Lynn
Published in: IEEE Symposium on Computers and Communications (ISCC), 2019

A Hybrid Fitness-Utility Algorithm for Improved Service Chain Placement

Author(s): Radhika Loomba, Thijs Metsch, Leonard Feehan, Joe Butler
Published in: 2018 IEEE Global Communications Conference (GLOBECOM), 2018, Page(s) 1-7
DOI: 10.1109/glocom.2018.8648033

Optimizing resource availability in composable data center infrastructures

Author(s): Ferreira, L., da Silva Rochay. E., Monteiroy, K., Santos, G., Silva, F., Kelner, J. Sadok, D., Bastos Filho, C., Rosati, P., Lynn, T. and Endo, P.
Published in: Proceedings of 2019 Ninth Latin-American Symposium on Dependable Computing (LADC)., 2020

Analysing dependability and performance of a real-world Elastic Search application

Author(s): Bendechache, M., Silva, I., Leoni Santos, G., Affonso Guedes, L., Svorobej, S., Noya, M., Eduardo Ares, M., Byrne, J., Endo, P. and Lynn, T.
Published in: 23rd International Symposium on Distributed Simulation and Real Time Applications (DS-RT 19) Cosenza, Italy, 2020

Modelling and Simulation of ElasticSearch using CloudSim

Author(s): Bendechache, M., Svorobej, S., Endo, P., Byrne, J. Noya Mariño, M. Eduardo Ares, M. and Lynn, T.
Published in: 9th Latin-American Symposium on Dependable Computing, 2020

RECAP (Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications): The Simulation Approach

Author(s): Endo, P.T., Filelis-Papadopoulos, C., Svorobej, S., Gourinovitch, A., Giannoutakis, K., Gravvanis, G., Tzovaras, D., Manimaran Elango, D., Byrne, J. and Lynn, T.
Published in: 7th European Conference on Service-oriented and Cloud Computing, 2018

A Modelling Language for Defining Cloud Simulation Scenarios in RECAP Project Context

Author(s): Cleber Matos de Morais, Patricia Endo, Sergej Svorobej, Theo Lynn
Published in: 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 2018, Page(s) 301-302
DOI: 10.1109/vlhcc.2018.8506544

RECAP - Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications

Author(s): Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim, Robbi Rahim
Published in: European Space Projects: Developments, Implementations and Impacts in a Changing World, 2017, Page(s) 75-86
DOI: 10.5220/0007901800750086

Workload Diffusion Modeling for Distributed Applications in Fog/Edge Computing Environments.

Author(s): T. Le Duc, M. Leznik, J. Domaschka, P-O. Östberg
Published in: Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering, ICPE 2020, 2020

Application, Workload, and Infrastructure Models for Virtualized Content Delivery Networks Deployed in Edge Computing Environments

Author(s): Thang Le Duc, Per-Olov Oestberg
Published in: 2018 27th International Conference on Computer Communication and Networks (ICCCN), 2018, Page(s) 1-7
DOI: 10.1109/icccn.2018.8487450

Power Shepherd: Application Performance Aware Power Shifting

Author(s): Jakub Krzywda, Ahmed Ali-Eldin, Eddie Wadbro, Per-Olov Ostberg, Erik Elmroth
Published in: 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), 2019, Page(s) 45-53
DOI: 10.1109/cloudcom.2019.00019

Modeling and Simulation of QoS-Aware Power Budgeting in Cloud Data Centers

Author(s): J. Krzywda, V. Meyer, M. Xavier, A. Ali-Eldin, P-O. Östberg, C. De Rose, E. Elmroth
Published in: Proceedings of the 28th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP 2020), 2020

ALPACA: Application Performance Aware Server Power Capping

Author(s): Jakub Krzywda, Ahmed Ali-Eldin, Eddie Wadbro, Per-Olov Ostberg, Erik Elmroth
Published in: 2018 IEEE International Conference on Autonomic Computing (ICAC), 2018, Page(s) 41-50
DOI: 10.1109/icac.2018.00014

A Review of Cloud Computing Simulation Platforms and Related Environments

Author(s): James Byrne, Sergej Svorobej, Konstantinos M. Giannoutakis, Dimitrios Tzovaras, P. J. Byrne, Per-Olov Östberg, Anna Gourinovitch, Theo Lynn
Published in: Proceedings of the 7th International Conference on Cloud Computing and Services Science, 2017, Page(s) 679-691
DOI: 10.5220/0006373006790691

Reliable capacity provisioning for distributed cloud/edge/fog computing applications

Author(s): Per-Olov Ostberg, James Byrne, Paolo Casari, Philip Eardley, Antonio Fernandez Anta, Johan Forsman, John Kennedy, Thang Le Duc, Manuel Noya Marino, Radhika Loomba, Miguel Angel Lopez Pena, Jose Lopez Veiga, Theo Lynn, Vincenzo Mancuso, Sergej Svorobej, Anders Torneus, Stefan Wesner, Peter Willis, Jorg Domaschka
Published in: 2017 European Conference on Networks and Communications (EuCNC), 2017, Page(s) 1-6
DOI: 10.1109/EuCNC.2017.7980667

A Preliminary Systematic Review of Computer Science Literature on Cloud Computing Research using Open Source Simulation Platforms

Author(s): Theo Lynn, Anna Gourinovitch, James Byrne, P. J. Byrne, Sergej Svorobej, Konstaninos Giannoutakis, David Kenny, John Morrison
Published in: Proceedings of the 7th International Conference on Cloud Computing and Services Science, 2017, Page(s) 565-573
DOI: 10.5220/0006351805650573

SQLR: Short-Term Memory Q-Learning for Elastic Provisioning

Author(s): Constantine Ayimba, Paolo Casari, Vincenzo Mancuso
Published in: Arxiv repository, 2019

Managing Distributed Cloud Applications and Infrastructure: A Self-Optimising Approach




The Cloud-to-Thing Continuum - Opportunities and Challenges in Cloud, Fog and Edge Computing