Periodic Reporting for period 2 - EGI-ACE (EGI Advanced Computing for EOSC) Reporting period: 2022-04-01 to 2023-06-30 Summary of the context and overall objectives of the project The EGI-ACE project delivers a distributed federated infrastructure that responds to the present and future needs of data-centric scientific computing for Open Science in Europe, bringing together compute facilities, data, scientific applications and software from the EGI Federation and research communities of pan-European relevance. EGI-ACE builds on (1) EOSC-hub project results; (2) data and applications from research collaborations, and (3) the operational capacity and expertise of the EGI e-infrastructure federation. The overall objective of EGI-ACE is "Implement the Compute Platform of the European Open Science Cloud and contribute to the EOSC Data Commons by delivering integrated computing, platforms, data spaces and tools as an integrated solution that is aligned with major European cloud federation projects and HPC initiatives.The EGI-ACE endeavour was launched in turbulent times - with limitations on physical meetings due to COVID-19, turbulence on energy prices, and limitations on open science and supply chains due to the war in Ukraine. Still, by the end, EGI-ACE not only succeeded in its activities and achieved its objectives but became one of the shining stars of the European Open Science Cloud project landscape. It reached unprecedented scales of compute delivery across diverse sectors and scientific disciplines. It empowered a large ecosystem of thematic services and data spaces made for open science. It have federated providers and user supporters from over 30 countries. The project not only brought EOSC to the next level but also expanded the EGI Federation and EGI Community with new members, new services, new partners and new users. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far EGI-ACE delivered the EOSC Compute Platform (ECP), a federated system of 36 compute and storage infrastructure and platform services that support diverse types of data processing and data analytics cases.The ECP includes High Throughput Compute (HTC) and Cloud Compute facilities, the latter for both Virtual Machine and Container execution, and was expanded in EGI-ACE with federated High Performance Compute sites in 2022. The platform layer of the ECP provides assistance for single sign-on, transfer and federation of distributed data, interactive computing, management of large numbers of jobs, orchestration of compute clusters, AI and machine learning tasks. The EOSC Compute Platform relies on a set of ‘Services for Federated Computing’ that enables its integration with the EOSC Core and reliable delivery to user communities The EGI-ACE consortium also delivers 17 thematic services (data spaces and processing platforms) that integrate data and applications from different scientific disciplines into the ECP for the scalable analysis and exploitation of scientific datasets. Moreover, 36 additional Thematic Services from outside the consortium are also supported by EGI-ACE services (18 out of these 36 are also registered in the EOSC-Exchange). Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) EGI-ACE has not only managed to succeed in its activities and achieved its objectives, but also has became one of the shining stars of the European Open Science Cloud project landscape by delivering unprecedented scales of compute across diverse sectors and scientific disciplines while empowering a large ecosystem of thematic services and data spaces made for open science. It has also federated providers and user supporters from over 30 countries. The project has not only brought EOSC to the next level, but also expanded the EGI Federation and EGI Community with new members, new services, new partners and new users. The impacts can be summarised as:- Impact towards Science: the project’s objective was achieved through the delivery of 36, free-at-point-of-use compute services through the EOSC Portal (19 Compute Platform services, 17 Thematic Data spaces and Processing Platforms, representing 9% of the total services (413) accessible in the EOSC Portal today. They attracted 189 access requests via the EOSC Marketplace (35% of all access orders), and received 42 access requests through the EGI-ACE Open Call, and were used by nearly 77,000 users.- Impact towards Research: the Consortium included 15 scientific groups who integrated application and scientific data with the EOSC Compute Platform resources and services, to establish and deliver 17 scalable data hosting and processing services (36 external services from outside of the Consortium) for the EOSC Exchange layer that served over 70,000 users.- Impact towards Collaboration: MoUs with e-infrastructures outside Europe, based on a shared vision of supporting compute-intensive Open Science with federated compute infrastructures as well as collaboration towards Research Infrastructures that includes in the Consortium RIs, expanded with 7 additional RIs are on the ESFRI roadmap. During EGI-ACE the EGI community also established a formal engagement structure towards industry: the EGI Digital Innovation Hub (EGI DIH): T-Systems and CloudFerro, MathWorks, 15 SMEs were attracted as users to the EGI-ACE services with the Open Calls.- Impact towards Infrastructure Innovation: the Cloud integration programme to increase both the capacity and the geographical footprint of this federated infrastructure with different level of integration: T-Systems and CloudFerro commercial providers, national cloud providers of Moldova, Latvia, Armenia, Georgia, OpenStack clouds from Greece (GRNET), Italy (INFN CNAF), Germany (GSI), Hungary (ELKH Cloud), Ireland (Walton Institute), China (CSTCloud), and South Africa (ILIFU). - Open National Infrastructures - Resource allocation approach: an innovative resource allocation model that combined multiple funding streams to reimburse the cost of services: (1) Virtual Access funding went directly from the EC to 33 providers. (73 Million CPU-hours of computing capacity delivered). (2) National and institutional funding from ministries and other types of funding agencies to providers to serve national or thematic user groups. (Over 100 providers providing 24 Million CPU-hours compute capacity. (3) Research community funding brought by the users to providers to consume services on a “pay-for-use” basis. (5 Million CPU-hours delivered). - Compute Continuum and Data Spaces: the infrastructure layer of the EOSC Compute platform initially built on compute cloud, compute container, and High Throughput Compute facilities. During EGI-ACE this layer was extended with High Performance Computing (HPC) systems. 4 HPC facilities have been brought into EGI CESGA (Spain), IICT-BAS (Bulgaria), LIP/INCD (Portugal), and TUBITAK (Turkey), and defined how the HPC-extension to the EOSC compute platform can be achieved. The project built 5 scientific Data Spaces on top of the Compute Platform- Impact towards Skills and Expertise : a network of shepherds and competence centres from technical experts in various institutes as well as delivered 26 webinars and 24 longer training events, attended by over 1,000 participants. The project delivered 18 FitSM training sessions in total, the result is 167 FitSM Foundation Certificates and 46 FitSM Advanced Certificates. - New Knowledge Gained and IP Generated: 69 publications that were submitted to scientific journals or conference proceedings and 2,815 citations. The project generated in total of 35 Intellectual Property assets (excluding the deliverables) including software code and know-how. EGI-ACE logo EGI-ACE fact and figures 3 Key Exploitable Results of EGI-ACE
