Periodic Reporting for period 1 - HiDALGO2 (HPC and Big Data Technologies for Global Challenges)
Période du rapport: 2023-01-01 au 2023-12-31
Climate change has long since been an undeniable phenomenon observed up close at many places on Earth. Climate change influences our everyday life, and thus has also an increasing impact on our quality of life. One of the key reasons is the dynamic development of societies which for decades significantly impacts the natural environment. This can be dramatically experienced nowadays by violent weather phenomena (e.g. storms, rainfalls). These problems are addressed by Global Challenges (GC). GC requires interdisciplinary expertise, and demands for solutions at scale due to their inherent complexity. Thus, HiDALGO2 aims to explore synergies between modelling, data acquisition, simulation, data analysis and visualisation along with achieving better scalability on current and future HPC and AI infrastructures to deliver highly-scalable solutions that can effectively utilise pre-exascale systems. The project focuses on five use cases from the environmental area: improving air quality in urban agglomerations, energy efficiency of buildings, renewable energy sources, wildfires and meteo-hydrological forecasting. The common feature of the modelling of the above simulations is the use of numerical analysis of fluid flows by Computational Fluid Dynamics (CFD) method, which is typically very compute-intensive. HiDALGO2 puts high emphasis on issues related to the scalability of solutions, the best adaptation of the software to the infrastructure (co-design) by using the appropriate benchmarking methodology and algorithmic optimization methods. This enables the efficient use of top-notch HPC systems to simulate complex structures with much greater accuracy not achievable for calculations using Cloud solutions. The quality of our solutions is assessed by uncertainty analysis carried out in ensemble runs mode. HiDALGO2 actively contributes to user communities from the EU by addressing the skills gap and sharing knowledge under organised specialised workflows and trainings.
Periodic Project Report of HiDALGO2 provides information about results for the first year of its execution (reporting period: 01/01/2023–31/12/2023). During the first year of its lifetime, HiDALGO2 has achieved the following main results:
• Providing all necessary mechanisms related to proper organization and communication in the project, such as: project bodies, mailing lists, common shared space for documents, WIKI and OpenProject. The project management process was formalized in the form of procedures and described in a deliverable D1.1 “Project Management Handbook” [1]. Moreover, an extensive coordination and decision-making support for the organisational and technical aspects have been provided throughout the whole first year.
• Definition of requirements and scenarios defining the scope of services in the HiDALGO2 project. They were collected iteratively from all partners with a focus on their dependencies and which partners need to coordinate with each other to achieve all project goals. Finding are presented in D2.1 [4].
• The HiDALGO2 architecture was established, and the initial status of the pilot application was identified (D5.3 [6]). The first version of the software was released, including proof of concept for performance optimization and pilot applications, the HiDALGO2 portal and an integrated workflow with HiDALGO2 tools.
• The project software was deployed on EuroHPC JU machines and integrated with CI/CD system delivered by the CASTIEL2 project.
• The first results in testing simulation applications on supercomputer systems were presented in report D3.1 [10].
• The methodology of the data management was carried out for the project and its results were documented (D1.5 [2]). Procedures for processing the data flows used in the project were established to ensure compliance with the legal and ethical framework at national and European level.
• The HiDALGO2 brand was established (D6.1 [5]), along with numerous communication channels (e.g. website, social media, newsletters, press releases) and activities in the area of presenting the HiDALGO2 offer (e.g. speeches, trainings), reported in D6.2 [11].
• Providing all necessary mechanisms related to proper organization and communication in the project, such as: project bodies, mailing lists, common shared space for documents, WIKI and OpenProject. The project management process was formalized in the form of procedures and described in a deliverable D1.1 “Project Management Handbook” [1]. Moreover, an extensive coordination and decision-making support for the organisational and technical aspects have been provided throughout the whole first year.
• Definition of requirements and scenarios defining the scope of services in the HiDALGO2 project. They were collected iteratively from all partners with a focus on their dependencies and which partners need to coordinate with each other to achieve all project goals. Finding are presented in D2.1 [4].
• The HiDALGO2 architecture was established, and the initial status of the pilot application was identified (D5.3 [6]). The first version of the software was released, including proof of concept for performance optimization and pilot applications, the HiDALGO2 portal and an integrated workflow with HiDALGO2 tools.
• The project software was deployed on EuroHPC JU machines and integrated with CI/CD system delivered by the CASTIEL2 project.
• The first results in testing simulation applications on supercomputer systems were presented in report D3.1 [10].
• The methodology of the data management was carried out for the project and its results were documented (D1.5 [2]). Procedures for processing the data flows used in the project were established to ensure compliance with the legal and ethical framework at national and European level.
• The HiDALGO2 brand was established (D6.1 [5]), along with numerous communication channels (e.g. website, social media, newsletters, press releases) and activities in the area of presenting the HiDALGO2 offer (e.g. speeches, trainings), reported in D6.2 [11].
Impact #1 Economic and Business Impact
The project's focus on these crucial aspects will take shape from M13, with the groundwork done in the first project phases (from 2018 to 2022). Although the task had not started in the first reporting period, proactive steps were already taken to engage with industry stakeholders. It is facilitated, amongst others, by active participation in industry-related activities organised by CASTIEL2. Participation in events and conferences, such as SIAM 2023, has further improved the visibility of the project and contact with potential business partners. The project team is also actively involved in the design of the CASTIEL2 marketplace, the so-called LinkHPC platform. This platform is developing into a marketplace where HiDALGO2 can offer services to external parties in the future. We are well prepared for the second reporting period, where the focus will be on creating business impact in the context of dissemination.
Impact #2 Societal Impact
The society will benefit from HiDALGO2 use cases as these use cases can teach us how to reduce risks and make better decisions for preventing them and for fighting them when they become real. Other areas could also be addressed in the future, in line with the current use cases, like food security (agriculture), healthcare or even economy. The activities of HiDALGO2 aim to achieve tangible societal impacts through active collaboration and engagement. The project has already made initial successes within the first reporting period. For example, the continuous involvement of SZE in FAIRMODE, the European Forum for Air Quality Modeling, and the collaboration between the Team for Renewable Energy Sources (RES) at PSNC and one of the largest Polish Distribution System Operators (DSO) are notable achievements. In the latter case, the collaboration focused on predicting damage to the overhead power grid in a major Polish city due to extreme weather conditions.
Impact #3 Scientific Impact
The project has also made progress in overcoming global challenges in its pilot applications. In the Urban Air Project (UAP), for example, the RedSIM implementation has been extended to multi-GPU and OpenMPI infrastructures, and work is underway on a digital twin prototype to be presented at HiPEAC 2024. For Urban Buildings (UB), developing algorithms, data structures, and visualisation tools for calculating solar protection masks at the district and city levels (e.g. Strasbourg and New York) was a significant success. In the area of renewable energy sources (RES), the coupling with the Weather Research and Forecasting Model (WRF) and the Eulerian Atmosphere Model (EULAG) was realised. Also, the project's commitment to visualisation aspects is underlined by the continued integration of the Unreal Engine with GIS data sources for immersive visualisation in VR environments, particularly in the wildfires pilot. This activity also helps reach out to society via more easy-to-understand visualisations for policymakers. Overall, HiDALGO2 has made significant progress towards our envisioned scientific impact in the first period.
The project's focus on these crucial aspects will take shape from M13, with the groundwork done in the first project phases (from 2018 to 2022). Although the task had not started in the first reporting period, proactive steps were already taken to engage with industry stakeholders. It is facilitated, amongst others, by active participation in industry-related activities organised by CASTIEL2. Participation in events and conferences, such as SIAM 2023, has further improved the visibility of the project and contact with potential business partners. The project team is also actively involved in the design of the CASTIEL2 marketplace, the so-called LinkHPC platform. This platform is developing into a marketplace where HiDALGO2 can offer services to external parties in the future. We are well prepared for the second reporting period, where the focus will be on creating business impact in the context of dissemination.
Impact #2 Societal Impact
The society will benefit from HiDALGO2 use cases as these use cases can teach us how to reduce risks and make better decisions for preventing them and for fighting them when they become real. Other areas could also be addressed in the future, in line with the current use cases, like food security (agriculture), healthcare or even economy. The activities of HiDALGO2 aim to achieve tangible societal impacts through active collaboration and engagement. The project has already made initial successes within the first reporting period. For example, the continuous involvement of SZE in FAIRMODE, the European Forum for Air Quality Modeling, and the collaboration between the Team for Renewable Energy Sources (RES) at PSNC and one of the largest Polish Distribution System Operators (DSO) are notable achievements. In the latter case, the collaboration focused on predicting damage to the overhead power grid in a major Polish city due to extreme weather conditions.
Impact #3 Scientific Impact
The project has also made progress in overcoming global challenges in its pilot applications. In the Urban Air Project (UAP), for example, the RedSIM implementation has been extended to multi-GPU and OpenMPI infrastructures, and work is underway on a digital twin prototype to be presented at HiPEAC 2024. For Urban Buildings (UB), developing algorithms, data structures, and visualisation tools for calculating solar protection masks at the district and city levels (e.g. Strasbourg and New York) was a significant success. In the area of renewable energy sources (RES), the coupling with the Weather Research and Forecasting Model (WRF) and the Eulerian Atmosphere Model (EULAG) was realised. Also, the project's commitment to visualisation aspects is underlined by the continued integration of the Unreal Engine with GIS data sources for immersive visualisation in VR environments, particularly in the wildfires pilot. This activity also helps reach out to society via more easy-to-understand visualisations for policymakers. Overall, HiDALGO2 has made significant progress towards our envisioned scientific impact in the first period.