Periodic Reporting for period 2 - DICE (Data Infrastructure Capacity for EOSC)
Période du rapport: 2022-04-01 au 2023-06-30
The DICE project has the purpose to enable a European storage and data management infrastructure for EOSC, providing generic services and building blocks to store, find, and access data in a consistent and persistent way. Specifically, DICE partners offer state-of-the-art data management services with more than 50 PB of storage capacity in total. The service and resource provisioning are accompanied by work to enhance the current service offering; the aim is to fill the gaps still present to the support of the entire research data life cycle: solutions are provided for increasing the quality of data and their re-usability, supporting long term preservation, managing sensitive data, and bridging between data and computing resources.
All services provided via DICE are being offered through the EOSC Portal and interoperable with EOSC Core via a lean interoperability layer to allow efficient resource. The partners have closely monitored the evolution of the EOSC interoperability framework and guidelines to comply with a) the rules of participation to onboard services into EOSC, and b) the interoperability guidelines to integrate with the EOSC Core functions.
The data services offered via DICE through EOSC are designed to be agnostic to the scientific domains in order to be multidisciplinary and to fulfil the needs of different communities. The consortium demonstrates the effectiveness of the service offering by integrating services with community platforms as part of the project and by engaging with new communities coming through EOSC. To this purpose three communities are directly part of the project to show case the usage of data management services in research data workflows: CompBioMed (Life Science), LOFAR (Astronomy) and ICOS (Atmosphere and Ecosystems Science).
All services provided via DICE are being offered through the EOSC Portal and interoperable with EOSC Core via a lean interoperability layer to allow efficient resource. The partners have closely monitored the evolution of the EOSC interoperability framework and guidelines to comply with a) the rules of participation to onboard services into EOSC, and b) the interoperability guidelines to integrate with the EOSC Core functions.
The data services offered via DICE through EOSC are designed to be agnostic to the scientific domains in order to be multidisciplinary and to fulfil the needs of different communities. The consortium demonstrates the effectiveness of the service offering by integrating services with community platforms as part of the project and by engaging with new communities coming through EOSC. To this purpose three communities are directly part of the project to show case the usage of data management services in research data workflows: CompBioMed (Life Science), LOFAR (Astronomy) and ICOS (Atmosphere and Ecosystems Science).
The DICE project has the following main objectives:
• To provide via the EOSC Portal state-of-the-art data services and significant capacity supported by the VA funding mechanism (WP7).
• To coordinate and manage the operations of the services and to integrate the operational tools to manage the offered services with the EOSC Core services (WP6 and WP3).
• To enhance the service provisioning by adding features for: easier integration with computing infrastructures; improving the integration, interoperability, and scalability of PID-systems; long-term preservation; and supporting of sensitive data management (WP4).
• Integration of DICE offered data services in community platforms to show case the advantages of the adoption of data management services (WP5).
The project has successfully achieved all its objectives and in particular:
- VA service offering:
o Provisioning of the available data management services under the VA mechanisms,
o Regular and efficient management of the requests made by users,
o Periodic monitoring of the allocated resources for the different installations,
- Operation management:
o Daily operation of the services and operational tools,
o Revision and update of the main SMS processes and procedures,
- Operational tools integration with the EOSC Core services:
o Monitoring and analysis of the interoperability guidelines as soon as they were available,
o Complete integration for monitoring, accounting, AAI, helpdesk, order management, and resource onboarding,
- New services features development:
o Development of solutions to integrate data services and HPC infrastructures (B2DROP with HPC-Systems and Jupyter-Hub at JSC and B2SAFE integration with HPC-Systems at CSC),
o Development of long-term preservation policies and template,
o Development of B2INST (a PID service for the identification of instruments that is based on B2HANDLE), integration of B2INST and ORCID with B2FIND and of B2FIND with OpenAIRE Explore,
o Development for the management of sensitive data of Secure B2SHARE and deployment in CSC Sensitive Data Infrastructure and in TSD research platform at the University of Oslo,
- Community platforms integration:
o Completed the integration of data management services with the research workflows related to the communities CompBioMed, LOFAR and ICOS,
o Collection of feedbacks and inputs from the Community Advisory Board,
- Dissemination and training:
o Very active participation to dissemination and outreach events as well as organisation of webinars describing the usage and features of the different services offered via DICE, collaboration in the organisation of the EUDAT conference in 2022,
o Organisation of hackathons in particular the one on Sensitive data and the EUDAT Summer School 2023,
- Collaboration with relevant other initiatives in EOSC landscape line EOSC Future, other INFRAEOSC-07 EC funded projects, EOSC Association and other EOSC-related projects.
• To provide via the EOSC Portal state-of-the-art data services and significant capacity supported by the VA funding mechanism (WP7).
• To coordinate and manage the operations of the services and to integrate the operational tools to manage the offered services with the EOSC Core services (WP6 and WP3).
• To enhance the service provisioning by adding features for: easier integration with computing infrastructures; improving the integration, interoperability, and scalability of PID-systems; long-term preservation; and supporting of sensitive data management (WP4).
• Integration of DICE offered data services in community platforms to show case the advantages of the adoption of data management services (WP5).
The project has successfully achieved all its objectives and in particular:
- VA service offering:
o Provisioning of the available data management services under the VA mechanisms,
o Regular and efficient management of the requests made by users,
o Periodic monitoring of the allocated resources for the different installations,
- Operation management:
o Daily operation of the services and operational tools,
o Revision and update of the main SMS processes and procedures,
- Operational tools integration with the EOSC Core services:
o Monitoring and analysis of the interoperability guidelines as soon as they were available,
o Complete integration for monitoring, accounting, AAI, helpdesk, order management, and resource onboarding,
- New services features development:
o Development of solutions to integrate data services and HPC infrastructures (B2DROP with HPC-Systems and Jupyter-Hub at JSC and B2SAFE integration with HPC-Systems at CSC),
o Development of long-term preservation policies and template,
o Development of B2INST (a PID service for the identification of instruments that is based on B2HANDLE), integration of B2INST and ORCID with B2FIND and of B2FIND with OpenAIRE Explore,
o Development for the management of sensitive data of Secure B2SHARE and deployment in CSC Sensitive Data Infrastructure and in TSD research platform at the University of Oslo,
- Community platforms integration:
o Completed the integration of data management services with the research workflows related to the communities CompBioMed, LOFAR and ICOS,
o Collection of feedbacks and inputs from the Community Advisory Board,
- Dissemination and training:
o Very active participation to dissemination and outreach events as well as organisation of webinars describing the usage and features of the different services offered via DICE, collaboration in the organisation of the EUDAT conference in 2022,
o Organisation of hackathons in particular the one on Sensitive data and the EUDAT Summer School 2023,
- Collaboration with relevant other initiatives in EOSC landscape line EOSC Future, other INFRAEOSC-07 EC funded projects, EOSC Association and other EOSC-related projects.
The DICE consortium has achieved the following at the end of the project:
* Increase the storage resources available to researchers by scaling-out the EOSC Portal.
* Enhance the data management service provisioning to cover the whole research data life cycle
* make the service provisioning fully interoperable with the EOSC core
* Engage research communities in the exploitation of DICE services
* Advance community platforms with data services offered through DICE
* Provide support for the management of sensitive data
With those objectives being achieved, the project made the following impacts:
1. Scale up the EOSC Portal through a growing catalogue to the broadest possible set of high-quality services and resources supporting the whole research life cycle from service providers across Europe and beyond.
2. More scientific communities across Europe are equipped and have access to state-of-the-art services for their research activities, increasing data-intensive research.
3. Facilitate Open Science practices across the research community in Europe with services to connect, share and re-use all type of research outputs, fostering collaboration and enhancing scientific discovery.
4. Foster synergies between pan-European e-infrastructures operators, leading to harmonised services, improved use of resources and economies of scale across Europe.
5. Support the collaboration in data provision and exchange across regional and national related infrastructures allowing the integration of data from a myriad of resources and research communities.
6. Coordinate and incentivise institutional and public actors so that they open up their services and resources to researchers across Europe, through a transparent and quality assured process.
* Increase the storage resources available to researchers by scaling-out the EOSC Portal.
* Enhance the data management service provisioning to cover the whole research data life cycle
* make the service provisioning fully interoperable with the EOSC core
* Engage research communities in the exploitation of DICE services
* Advance community platforms with data services offered through DICE
* Provide support for the management of sensitive data
With those objectives being achieved, the project made the following impacts:
1. Scale up the EOSC Portal through a growing catalogue to the broadest possible set of high-quality services and resources supporting the whole research life cycle from service providers across Europe and beyond.
2. More scientific communities across Europe are equipped and have access to state-of-the-art services for their research activities, increasing data-intensive research.
3. Facilitate Open Science practices across the research community in Europe with services to connect, share and re-use all type of research outputs, fostering collaboration and enhancing scientific discovery.
4. Foster synergies between pan-European e-infrastructures operators, leading to harmonised services, improved use of resources and economies of scale across Europe.
5. Support the collaboration in data provision and exchange across regional and national related infrastructures allowing the integration of data from a myriad of resources and research communities.
6. Coordinate and incentivise institutional and public actors so that they open up their services and resources to researchers across Europe, through a transparent and quality assured process.