Periodic Reporting for period 2 - INDIGO-DataCloud (INtegrating Distributed data Infrastructures for Global ExplOitation)
Reporting period: 2016-07-01 to 2017-09-30
The project addressed several technological issues that prevented easy and efficient exploitation of Cloud resources in many scientific domains, such as:
• missing consistent authentication and authorization policies across both applications and infrastructures;
• the difficulty of actually finding and using data and computing resources necessary for a given problem;
• the problem of negotiating and guaranteeing clear Quality of Service policies;
• the issue of expressing in a simple way high-level requirements that go well beyond the simple concept of a “Virtual Machine”;
• the trouble in expressing complex scientific workflows in Cloud infrastructures;
• the partial solutions currently available to integrate legacy applications into Cloud-based scientific portals, mobile appliances, or to write complex web-based front-ends exploiting advanced Cloud features;
• the problematic and often un-scalable deployment of applications necessitating Cloud resources;
• the difficulty in interfacing with both public and private Cloud infrastructures, avoiding proprietary lock-ins and licensing issues.
INDIGO-DataCloud tackled these problems on the one hard developing building blocks, or tools, that respond to the requirements of many scientific communities; on the other hand, applying these tools to concrete scientific use cases and applications, deploying them to both public and private e-infrastructures. This results in faster results and in better and easier use of data and compute resources across Europe and elsewhere. These high-level objectives are summarized by the INDIGO motto: “Better Software for Better Science”.
In the area of dissemination and exploitation, INDIGO collaborated with several commercial companies such as ATOS, T-Systems, IBM, INDRA and Santer-REPLY to facilitate the adoption and enhancement of INDIGO components. An ambitious goal the project set for itself was to promote INDIGO solutions as key components of a future European Open Science Cloud (EOSC): in fact, INDIGO is one the three projects that, together with EGI-Engage and EUDAT2020, coordinated the successful preparation and submission of the EOSC-hub project, which will contribute to the EOSC implementation by enabling seamless and open access to a system of research data and services provided across countries and multiple disciplines. In EOSC-hub, INDIGO nominates the overall Technical Coordinator of the project; many INDIGO components at Technology Readiness Level (TRL) 8 or above will also find place in the EOSC unified service catalogue, and INDIGO solutions are at the foundation of several EOSC-hub Thematic Services and Competence Centers. In addition to EOSC-hub, two additional Horizon2020 projects that directly derive from INDIGO-DataCloud were also submitted and positively evaluated, i.e. eXtreme-DataCloud (XDC) and DEEP-HybridDataCloud. These two follow-on projects will introduce novel features and services to existing TRL6 INDIGO components in many areas, with the objective of bringing them to TRL8 and eventually including them into the EOSC service catalogue.
These advancements are fully described in the INDIGO Service Catalogue and in scientific applications making use of INDIGO components, both described at the project website. The key impact of the project is toward easy and efficient usage of both public and private compute & data resources, in the development of cost-efficient, state-of-the-art scientific services and applications that are interoperable across diverse infrastructures, and ultimately toward producing results in many scientific domains in a faster, more effective way. INDIGO-developed solutions have for instance enabled new advances in understanding how the basic blocks of matter (quarks) interact, using supercomputers, how new molecules involved in life work, using GPUs, or how complex new repositories to preserve and consult digital heritage can be easily built.