Skip to main content

eXtreme DataCloud

Deliverables

Periodic Progress Report - update

A report describing the project achievements, the result of the development and testing activities. It includes the exploitation activities results analysis and will describe the progress towards the fulfilment of the main objectives. It will contain an investigation of the results as they appear from the KPIs and progress metrics. Produced yearly.

Final Report

This report contains a consolidated view of the results of all tasks, with particular focus on the status of the Pilot testbeds, their integration in external e-Infrastructures like the EGI & EINFRA-12 and EOSC. It highlights how use cases requirements were met, and challenges regarding handling Big Data and extremely heterogeneous data, metadata types and formats were overcome. It contains also the validation of the readiness level of the new features together with plans for exploitation of the project results.

Software Lifecycle Management and Pilot testbeds - Initial plan

This document describes the Software Lifecycle Management process - the Quality Assurance, Maintenance, Release and Support processes, the roles and responsibilities. Definition of the architectures of the Pilot infrastructures according to use cases to be supported. It includes the selection of tools to support the task 3.1 and the architecture of the Continuous Integration/Delivery infrastructure supporting the DevOps model. It describes the plan for liaison with external infrastructures and resource providers.

Unified data management platform evaluation report

This deliverable will describe the evaluation results of the integrated unified data management platform including functional as well as non-functional aspects, in particular analysis of the scalability potential of the solution.

Risk analysis and contingency plan - update

Analyses the possible risks that can constitutes an obstacle to the successful fulfilment of the project objectives or that can result in a significant delay in time for some of the project developments. Define re-planning or contingency plans for every identified risks. Created at Month 4 and updated after 11 months.

Performance and scalability evaluations of the prototype implementation

It provides a status report of the prototype implementation and an evaluation of its performance

Periodic Progress Report

A report describing the project achievements, the result of the development and testing activities. It includes the exploitation activities results analysis and will describe the progress towards the fulfilment of the main objectives. It will contain an investigation of the results as they appear from the KPIs and progress metrics. Produced yearly.

Extended list of requirements

The deliverable is linked to a specific event (M2.2) and an eventual report. The goal of the event is to join both User Communities representatives and developers in order to ensure that the initial list of functionalities have been taken into account. Additionally, and based on the evolving user experience and the findings of task T2.2 (reflected in D2.2), an extended list of functionalities is provided to be taken into account in future releases and developments.

General Architecture

Describes the architecture of the data management solutions and tools that will be developed by the project taking into account the use case requirements analyzed in details and prioritized by Work Package 2 after the first six month of activity. Provides the high-level view of the technical development plan for all the services, summarizing the detailed information provided by WP4 and WP5 in D4.1 and D5.1. It contains an initial status assessment and the advancement with respect to the state of the art of the envisaged solutions. Produced by WP1 representing the Technical Coordination Board.

Project Quality Assurance and Progress Monitoring

Describes the project internal quality assurance plan and all procedures to enforce decisions, monitor the status of the project and react to identified issues. This does not include the Software Quality Assurance process, which is specifically defined and monitored by Work Package 3.

Annual Report on Software Lifecycle Management and Pilot testbeds

This report contains a consolidated view of the results of all the tasks with particular focus on the compliance with the established processes and procedures, status of the pilot infrastructure with respect to the use cases addressed in the reporting period.

Risk analysis and contingency plan

Analyses the possible risks that can constitutes an obstacle to the successful fulfilment of the project objectives or that can result in a significant delay in time for some of the project developments. Define re-planning or contingency plans for every identified risks. Created at Month 4 and updated after 11 months.

Evaluation report on performance and scalability of the final release

It provides a final report on the implementation activities and a performance evaluation, including scalability, of all the WP4 released components. The measurements will be performed on the testbed in coordination with WP2.

General Architecture for the Orchestration and policy driven XDC solutions

It defines the general architecture of all tasks products and includes detailed work program for the rest of the project

XDC Solutions for EOSC research communities

The deliverable is linked to a specific event and an eventual report. This workshop aims to find synergies among the XDC project itself and other initiatives and pilots from the EOSC, and tries to promote the use of the XDC solutions to wider environments involving extremely large and heterogeneous datasets and data sources in a federated way. The event also will define how the different Use Cases matches in the EOSC environment and how other Research Communities can learn and adapt the solutions to their cases. Stakeholders of the main European e-Infrastructures will be invited.

Dissemination Plan

This report summarize the work done in task T2.4, including the objectives of the dissemination activities, target identification, media and tools to be used and the metrics to assess the effectiveness of the dissemination. It will also include details about the training and potential targets. The deliverable includes the specific section in the project website.

Use Cases Validation and Final satisfaction metrics/Confirmation of scalability for Use Cases

This deliverable starts from the definition of tests and satisfaction metrics described in the Deliverable D2.5. Those tests will be implemented and performed in the testbed provided by WP3. The report summarizes the results of those tests as well as the general user satisfaction at the end of the project. Finally, it verifies, after having performed scalability test, that the solutions provided can be exploited in wider environments.

Unified data management layer architecture

This deliverable will define the architecture of the unified data management system architecture based on the feedback from the use case analysis in WP2 and low-level data management design in WP4.

Unified data management platform integrated version

This deliverable will describe the implementation of the integrated unified data management platform.

Unified data management platform first prototype

This deliverable will describe the functionality, the implementation details and initial tests results of the first unified data management prototype.

Identification of new functionalities

This report identifies the new functionalities to be added to the existent services and tools, taking into account the Use Cases described by the Research Communities represented as well as the findings of task T2.1. It serves as input for the WP4 and WP5, indicating the missed features to be implemented during the project. D2.1 includes a general description of the communities involved in the project that aims to define the complexity and the heterogeneity of the environment. It also includes the definition of the different Use Cases with different types of details. In particular, the Use Case description identifies and analyzes the different data sources involved in terms of size, data types and/or formats, as well as the workflows and definition of the data lifecycle. Metadata management issues and other key actions like pre-processing needs, encryption requirements or data caching will have special consideration. The description should also define other transversal requirements like the use of Authentication and Authorization mechanisms or security and license needs. The analysis of Data Management Plans is included aiming at finding both synergies and gaps among the different communities. The document reports the goals oriented to Open Science and Data, so that requirements in terms of Digital Identifiers for digital object (including datasets, software and others) are included. All the Use Cases take into account the different available products and to pre-defines those that could match to their needs.

Enhance Data Management Best practices

This report aims to analyze the state-of-art of best practices in data and metadata management, including those proposed by the Research Data Alliance, FORCE11 (FAIR+R principles) and other initiatives and organizations. All the initiatives related to the EOSC will have special consideration. D2.3 identifies the advances in terms of Data Management Planning, including the use of dynamic or machine actionable DMPs that could be adopted by the communities or by the project services. Finally, this report defines which of those best practices will be addressed within the context of the project and in collaboration with other initiatives and how they can contribute to the development of the EOSC.

Test Cases: Integration

This deliverable reports the activities performed in Task 2.3, including the defined tests to measure the different developed solutions. The report also describes different metrics to measure the user satisfaction. D2.5 defines a set of scalability tests that will assure that all the solutions developed will be useful in bigger environments.

Workshop: Extended list of requirements for XDC

The goal of the event is to join both User Communities representatives and developers in order to ensure that the initial list of functionalities have been taken into account. Additionally, and based on the evolving user experience and the findings of task T2.2 (reflected in D2.2), an extended list of functionalities is provided to be taken into account in future releases and developments.

Workshop - XDC Solutions for EOSC Research Communities

This workshop aims to find synergies among the XDC project itself and other initiatives and pilots from the EOSC, and tries to promote the use of the XDC solutions to wider environments involving extremely large and heterogeneous datasets and data sources in a federated way. The event also will define how the different Use Cases matches in the EOSC environment and how other Research Communities can learn and adapt the solutions to their cases. Stakeholders of the main European e-Infrastructures will be invited.

Data Management Plan

The analysis of Data Management Plans aiming at finding both synergies and gaps among the different community’s needs. The document reports the goals oriented to Open Science and Data, requirements in terms of Digital Identifiers for digital object (including datasets, software and others) are included.

Searching for OpenAIRE data...

Publications

Smart Policy Driven Data Management and DataFederations

Author(s): Patrick Fuhrmann, Marica Antonacci, Giacinto Donvito, Oliver Keeble, Paul Millar
Published in: Proceedings of Sciecne, 2018

Storage for Advanced Scientific Use-Cases and Beyond

Author(s): Paul Millar, Olufemi Adeyemi, Gerd Behrmann, Patrick Fuhrmann, Vincent Garonne, Dmitry Litvinsev, Tigran Mkrtchyan, Albert Rossi, Marina Sahakyan, Jurgen Starek
Published in: 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), 2018, Page(s) 651-657
DOI: 10.1109/pdp2018.2018.00109

Advancements in data management services for distributed e-infrastructures: the eXtreme-DataCloud project

Author(s): COSTANTINI Alessandro, CESINI Daniele, DONVITO Giacinto, DUMA Doina Cristina, MATTHEW Viljoen, BATTAGLIA Serena, POIREAU Vincent, DELL’AGNELLO Luca, KEEBLE Oliver, LEMRANI Rachid, OHMANN Christian, MARCO DE LUCAS Jesus, DUTKA Lukasz, FUHRMANN Patrick, AGUILAR GOMEZ Fernando
Published in: 2019

Water quality forecasting system

Author(s): Garcia, Daniel; Aguilar, Fernando; Castrillo, Maria; Marco, Jesús; Monteoliva, Agustin
Published in: Issue 1, 2018

Hydrodynamic and water quality modelization of Cuerda del Pozo reservoir

Author(s): Castrillo, Maria; Garcia, Daniel; Aguilar, Fernando; Marco, Jesús; Monteoliva, Agustín
Published in: Issue 1, 2018

A set of common software quality assurance baseline criteria for research projects

Author(s): Orviz Pablo, López García Álvaro, Duma Doina Cristina, Donvito, Giacinto, David Mario, Gomes Jorge
Published in: 2017

ECRIN Clinical Research Metadata Schema Version 3.0 (November 2019)

Author(s): Canham, Steve
Published in: Issue 2, 2019
DOI: 10.5281/zenodo.1312538

ECRIN Clinical Research Metadata Schema Version 2.2 (February 2019)

Author(s): Canham, Steve; Ohmann, Christian
Published in: Issue 2, 2019
DOI: 10.5281/zenodo.3534313

Estimation of high frequency nutrient concentrations from water quality surrogates using machine learning methods

Author(s): María Castrillo, Álvaro López García
Published in: Water Research, Issue 172, 2020, Page(s) 115490, ISSN 0043-1354
DOI: 10.1016/j.watres.2020.115490

DataCloud infrastructure to manage FAIR environmental data

Author(s): F. Aguilar
Published in: Journal of Instrumentation, Issue 15/04, 2020, Page(s) C04009-C04009, ISSN 1748-0221
DOI: 10.1088/1748-0221/15/04/c04009