Community Research and Development Information Service - CORDIS

H2020

DICE Report Summary

Project ID: 644869
Funded under: H2020-EU.2.1.1.3.

Periodic Reporting for period 2 - DICE (Developing Data-Intensive Cloud Applications with Iterative Quality Enhancements)

Reporting period: 2016-02-01 to 2017-01-31

Summary of the context and overall objectives of the project

Recent years have seen the rapid growth of interest for cloud applications built on top of Big Data technologies such as MapReduce/Hadoop, NoSQL databases, cloud-based storage, and stream processing systems. However, there is still a visible shortage of software engineering models, methods and tools for developing data-intensive software systems. Moreover, the rush to hit the Big Data market with new products leads companies to reduce the attention they pay to quality aspects, which are yet critical to avoid project failures. This issue is exacerbated by the fact that quality engineering of data-intensive software systems is still in its infancy, making it difficult today to analyse, predict and guarantee efficiency, reliability and safety properties for these applications.
DICE aims at addressing these challenges by defining a novel framework for quality-driven development of data-intensive applications. The DICE methodology will cover quality assessment, architecture enhancement, testing and agile delivery, relying on principles of the emerging DevOps paradigm. At a high-level, the DICE paradigm aims at:
- Tackling skill shortages and learning curves in quality-driven development and Big Data technologies through open source development tools, models, and methods.
- Shortening the time to market for data-intensive applications that meet quality requirements, reducing costs for independent software vendors (ISVs) and increasing value for end-users.
- Reducing the number and the severity of quality incidents by iteratively learning the quality-levels of the application at runtime, feeding this information back to developers.

To achieve these goals, DICE proposes a model-driven engineering (MDE) framework for Big Data applications based on UML, and contextualizes such framework into a DevOps process. The DICE framework features a quality engineering tool chain for simulation and verification of efficiency (cost and performance), reliability, and safety properties of the application. Using a DevOps-inspired approach, the DICE framework allows rapid integration and delivery of novel prototypes. The DICE approach uses monitoring and feedback analysis techniques to deeply analyze monitoring data collected during testing and prototyping in order to identify quality incidents and accelerate the iterative enhancement of the application across release cycles.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The technical work done since project start has been primarily concerned with the following aspects:

- Analysis of the state of the art in software engineering and technologies for data-intensive applications (cf. deliverable D1.1).

- Specification from the experience gathered in D1.1 of a set of technical requirements for the DICE WPs (cf. D1.2, D1.4). This work has also identified an initial subset of technologies to prioritize (including Hadoop/MapReduce, Storm, Spark, Cassandra, MongoDB) and the definition of requirements for the demonstrators (D6.1).

- Specification of a high-level architecture for the DICE framework and associated release of the DICE IDE (D1.4).

- Consolidated specification of the DICE Models and DICE Profile that will be used by developers to create and refine data-intensive applications through the DICE IDE (D2.2).

- DICE Simulation Tool to predict quality characteristics of software systems in the early design stages of a data- intensive application (D3.3), and the DICE Verification Tool, which verifies formal properties relatively to safety and reachability (D3.6). Quality assessment relies on model-to-model transformations defined (D3.1).

- Initial and consolidated version of the Deployment modelling tool to produce viable and deployable orchestration blueprints featuring the TOSCA standard (D2.3) alonsgide the DICE deployment and configuration tools (D5.2).

- The DICE Quality Testing tool (D5.4) provides the means for the developers to generate controlled load to their Storm application and thus assess the performance of the DIA being developed.

- The Fault Injection tool (D5.6) simulates VM-level faults such as saturation of resources and failed services, enabling a controlled way of validating the DIA's resilience to faults.

- Initial tool deployment and validation from all three demonstrators and evaluation of tools according to a defined set of KPIs (D6.2).

Moreover the following non-technical activities have been performed (cf. D7.5, D7.6, D8.4, D8.5, D8.7, D8.8):

- A sustained dissemination effort across scholarly publications. Communications via website (www.dice-h2020.eu) and social media channels, and participation to several industrial events. This includes guest blogging on the Cloudify blog.

- Collaboration with the SE4SA software engineering cluster and EU projects such as MODAClouds and CACTOS and collaboration with MIKELANGELO. Collaboration with the SPEC RG DevOps working group, comprising joint organization of two edition of the QUDOS workshop (qudos2015.fortiss.org at ESEC/FSE 2015, qudos2016.fortiss.org at ISSTA 2016).

- Standardization activities through the TOSCA working group. TOSCA is a cloud topology and orchestration specification language that provides the baseline for the DICE DDSM models. In January 2017 XLAB joined the TOSCA TC, further strengthening the DICE’s position in the TOSCA standardization. Export of monitoring data into the OSLC standard to allow external DevOps tools to reuse the D-Mon monitoring platform.

- Identification of business value propositions, exploitation and sustainability strategies (D7.6). Establishing formal partnership contracts with the Multiclouds Alliance for supporting the sustainability of the tools after the end of the project.

- Definition of a research data management policy focussed on the European portal Zenodo (https://zenodo.org). Definition of the internal collaboration platform. Implementation of quality management and ethics issues management practices.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

DICE will deliver innovative development methods and tools to strengthen the competitiveness of small and medium European ISVs in the market of business-critical data-intensive applications. The barriers that DICE intends to break are the shortage of methods to express data-aware quality requirements in model-driven development and the ability to consistently consider these requirements throughout the tool-chain of quality analysis, testing, and deployment. Existing methodologies and tools provide these capabilities for traditional enterprise software systems and cloud-based applications, but when it comes to increasingly popular technologies such as, e.g., Hadoop/MapReduce, Spark, Storm, Cassandra, it is not possible to adopt a quality-driven software engineering approach yet. DICE intends to deliver this capability, providing a quality-driven development environment for data-intensive applications.
In the design of a data-intensive applications, existing software engineering approaches face a number of limitations, even if one considers the basic specification of requirements. For example, it is possible with MDE to express entity-relationship models, basic dependencies between components and data, field types and values, and data semantics. In the operations of a data-intensive application, DICE offers methods for deployment, monitoring, testing, configuration and anomaly detection for the aforementioned data-intensive technologies.

Related information

Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top