Periodic Reporting for period 2 - ChEESE (Centre of Excellence for Exascale in Solid Earth)
Reporting period: 2020-05-01 to 2022-03-31
Solid Earth (SE) sciences address fundamental problems in understanding the Earth. Exascale computing will allow advances on, at least, three fronts:
1. On explicitly solving problems that, traditionally, have been parameterized because are unaffordable in terms of size, model resolution, complex physical couplings, and huge amounts of data involved.
2. On solving currently affordable problems but with much less computing time. This is critical in scenarios of early warning and emergency management.
3. Introducing a probabilistic framework through a large ensemble of model realizations, model data assimilation, and uncertainty quantification.
SE sciences have significant scientific and socioeconomic implications, including geohazards, hydrocarbon and energy resource exploration, or containment of underground waste and carbon sequestration. From a social point of view, it is important to foster scientific, engineering, and information technology innovation for a better risk management of geohazards. This is relevant to support the preparation of EU policies on risk mitigation and adaptation and assist in the development of civil protection capabilities to protect EU citizens from geohazards.
The Center of Excellence for Exascale in Solid Earth (ChEESE; GA No 823844) has run from November 2018 to March 2021. ChEESE has prepared 10 Community flagship codes in the areas of computational seismology, tsunamis, volcanology and magnetohydrodynamics for the present pre-exascale and the upcoming exascale hardware architectures. The flagship applications have been used together with workflows to implement 12 Pilot Demonstrators (PDs) addressing exascale capability and capacity challenges in the area of SE. The 8 PDs reaching a TRL equal or higher than 5 during the second half of the project have also been used to enable and validate services on critical aspects of geohazards such as urgent computing during emergency situations, probabilistic hazard assessment, early warning, and Earth’s subsurface imaging. This has been done in close collaboration with members of the Industry and Users Board (IUB), which have also furnished the appropriate use-cases and requirements to perform 14 test cases and 6 live exercises in which services ran in operational mode. ChEESE has also been very successful in integrating the community around HPC and leveraging the growth of an ecosystem of “child” projects and initiatives that benefit from (and synergize with) the CoE.
1. On explicitly solving problems that, traditionally, have been parameterized because are unaffordable in terms of size, model resolution, complex physical couplings, and huge amounts of data involved.
2. On solving currently affordable problems but with much less computing time. This is critical in scenarios of early warning and emergency management.
3. Introducing a probabilistic framework through a large ensemble of model realizations, model data assimilation, and uncertainty quantification.
SE sciences have significant scientific and socioeconomic implications, including geohazards, hydrocarbon and energy resource exploration, or containment of underground waste and carbon sequestration. From a social point of view, it is important to foster scientific, engineering, and information technology innovation for a better risk management of geohazards. This is relevant to support the preparation of EU policies on risk mitigation and adaptation and assist in the development of civil protection capabilities to protect EU citizens from geohazards.
The Center of Excellence for Exascale in Solid Earth (ChEESE; GA No 823844) has run from November 2018 to March 2021. ChEESE has prepared 10 Community flagship codes in the areas of computational seismology, tsunamis, volcanology and magnetohydrodynamics for the present pre-exascale and the upcoming exascale hardware architectures. The flagship applications have been used together with workflows to implement 12 Pilot Demonstrators (PDs) addressing exascale capability and capacity challenges in the area of SE. The 8 PDs reaching a TRL equal or higher than 5 during the second half of the project have also been used to enable and validate services on critical aspects of geohazards such as urgent computing during emergency situations, probabilistic hazard assessment, early warning, and Earth’s subsurface imaging. This has been done in close collaboration with members of the Industry and Users Board (IUB), which have also furnished the appropriate use-cases and requirements to perform 14 test cases and 6 live exercises in which services ran in operational mode. ChEESE has also been very successful in integrating the community around HPC and leveraging the growth of an ecosystem of “child” projects and initiatives that benefit from (and synergize with) the CoE.
The 10 flagship codes have been audited in collaboration with the PoP2 CoE. Dedicated code optimisation and tuning actions have been taken and an increased number of ChEESE applications are already suitable to the actual pre-Exascale GPU architectures. 7 flagship codes have been fully or partially ported to GPU environments and all codes initially labeled as scalability level 3 are at level 2 at the end of the project. In terms of co-design, 4 mini-apps and 2 synthetic benchmarks have been developed for testing on different typologies of emerging hardware. A Workflow Management System (WMS-light) has been defined to orchestrate the scientific workflows associated with the PDs. This also includes some pre-process components like the topological mesh partitioner amik library. A Data Management Plan (DMP) has been defined to ensure a FAIR management of data and metadata in all the PDs, and the scenarios and roadmap for running Exascale testbeds has been established to run full-scale simulations. The 12 PDs have been implemented, 1 achieved TRL 5, 3 TRL 6, 1 TRL 7, 1 TRL 8 and 2 of them have already reached a TRL of 9. PDs with a higher potential in a transition phase towards validation in an operational setting and enabling different typologies of services. 8 Pilot Demonstrators were available as services to a broader user community and to involve the geophysical community and the non-scientific stakeholders in the proper definition of the service validation process, especially focusing on hazard and risk assessment goals. It has also validated and implemented some of the Pilot Services (those achieving TRL 7-8) in the HPC ecosystem and evaluated the resources to enable their future operational deployment. Results include:
● Support to PRACE grants and allocation of HPC resources for pilots (217 M core hours granted)
● Definition of service typologies
● End-users from the IUB have been involved in the co-design of services by defining, in synergy with the ChEESE teams, the appropriate use-cases and requirements for the validation in 14 use cases and 6 user-driven validation exercises
● Faster than real-time tsunami simulation service. The PD2 exploited operationally (TRL=9) by IGN (the Spanish Instituto Geografico Nacional), ARISTOTLE-eENHSP delivering to the European Emergency Response Coordination Centre (ERCC), and the ARISTOTLE tsunami service is integrated in the SPADA (Scientific Products Archiving and Document Assembly) IT platform.
● Urgent computing during La Palma eruption. The PD12 ran operationally (TRL=9) during the eruption. Ensemble-based (scenarios) ash dispersal forecasts ran @MN4 from 19 Nov to 13 Dec 2021 and delivered daily (at 8:00 am LT) to the scientific committee of the PEVOLCA and to the civil protection authorities for real operational decision-making
ChEESE has organized 15 training courses (including 5 PATC). The KPIs for scientific and social impacts of the project are evolving very satisfactorily, and a proposal to organize a European Geosciences Union (EGU) Galileo conference has been granted.
● Support to PRACE grants and allocation of HPC resources for pilots (217 M core hours granted)
● Definition of service typologies
● End-users from the IUB have been involved in the co-design of services by defining, in synergy with the ChEESE teams, the appropriate use-cases and requirements for the validation in 14 use cases and 6 user-driven validation exercises
● Faster than real-time tsunami simulation service. The PD2 exploited operationally (TRL=9) by IGN (the Spanish Instituto Geografico Nacional), ARISTOTLE-eENHSP delivering to the European Emergency Response Coordination Centre (ERCC), and the ARISTOTLE tsunami service is integrated in the SPADA (Scientific Products Archiving and Document Assembly) IT platform.
● Urgent computing during La Palma eruption. The PD12 ran operationally (TRL=9) during the eruption. Ensemble-based (scenarios) ash dispersal forecasts ran @MN4 from 19 Nov to 13 Dec 2021 and delivered daily (at 8:00 am LT) to the scientific committee of the PEVOLCA and to the civil protection authorities for real operational decision-making
ChEESE has organized 15 training courses (including 5 PATC). The KPIs for scientific and social impacts of the project are evolving very satisfactorily, and a proposal to organize a European Geosciences Union (EGU) Galileo conference has been granted.
ChEESE is progressing on approaching the SE Community towards Exascale, tackling societal georisks with an unprecedented approach and acting as a pole of attraction to gather actors with very different backgrounds around HPC/HPDA.
In terms of scientific progress:
● FWI and high-resolution subsurface imaging to model seismic wave propagation at unprecedented high frequencies in complex geological settings (ECC1 and ECC2).
● Seismic hazard studies based upon physically consistent models.
● Automated array-based statistical detection and restoration of seismic slow-earthquakes (ECC15).
● Physical coupling of earthquakes and tsunamis in 3D.
● In magneto-hydrodynamics, investigate the conditions that lead to geomagnetic reversals, and the properties of the magnetic field during the phenomenon (ECC5).
● For volcanic plumes, characterize the microphysics and the non-linear coupling between turbulence scales (ECC7).
● High-resolution volcanic ash dispersal simulations for hazard assessment and operational forecast using data assimilation (ECC8).
● The portal and repository of HPC codes and toolkits coordinated with (and integrated to) EPOS.
In terms of innovative solutions and social impact:
● Establishment of a pan-European urgent computing system (with EuroHPC) including near real-time seismic scenarios (ECC3) and tsunami simulation in the framework of tsunami Early Warning Systems (ECC11).
● Physics-based probabilistic hazard assessment for earthquakes, tsunamis and volcanoes (ECC4, ECC10, ECC14).
● Probabilistic Tsunami Forecast for early warning and rapid post-event assessment (ECC13).
● Geophysical exploration. Subsurface imaging for identification and management of energy/mineral reservoirs, storage sites, and their monitoring through time.
● Volcanic ash dispersal forecasts at unprecedented resolution (few km) based on satellite data assimilation and ensemble forecast.
In terms of scientific progress:
● FWI and high-resolution subsurface imaging to model seismic wave propagation at unprecedented high frequencies in complex geological settings (ECC1 and ECC2).
● Seismic hazard studies based upon physically consistent models.
● Automated array-based statistical detection and restoration of seismic slow-earthquakes (ECC15).
● Physical coupling of earthquakes and tsunamis in 3D.
● In magneto-hydrodynamics, investigate the conditions that lead to geomagnetic reversals, and the properties of the magnetic field during the phenomenon (ECC5).
● For volcanic plumes, characterize the microphysics and the non-linear coupling between turbulence scales (ECC7).
● High-resolution volcanic ash dispersal simulations for hazard assessment and operational forecast using data assimilation (ECC8).
● The portal and repository of HPC codes and toolkits coordinated with (and integrated to) EPOS.
In terms of innovative solutions and social impact:
● Establishment of a pan-European urgent computing system (with EuroHPC) including near real-time seismic scenarios (ECC3) and tsunami simulation in the framework of tsunami Early Warning Systems (ECC11).
● Physics-based probabilistic hazard assessment for earthquakes, tsunamis and volcanoes (ECC4, ECC10, ECC14).
● Probabilistic Tsunami Forecast for early warning and rapid post-event assessment (ECC13).
● Geophysical exploration. Subsurface imaging for identification and management of energy/mineral reservoirs, storage sites, and their monitoring through time.
● Volcanic ash dispersal forecasts at unprecedented resolution (few km) based on satellite data assimilation and ensemble forecast.