Periodic Reporting for period 3 - ESiWACE (Excellence in SImulation of Weather and Climate in Europe)
Reporting period: 2018-09-01 to 2019-08-31
ESiWACE has demonstrated and improved scalability of some of Europe’s leading weather and climate models on world class HPC systems and is prototyping new tools and methods for management of climate and weather data in the exascale era. However, we are still far from being able to run a 1 km model in operational mode even if some of the fastest supercomputers that are currently available are used. The necessary steps to achieve this goal have been documented in the ESiWACE sustainability plan and a roadmap. The second phase of the project, ESiWACE2, will continue the work and follow up on the roadmap.
1) Scalability: Global High-Resolution Demonstrators
Very high-resolution simulations were successfully developed based on the atmosphere model IFS, ICON and the ocean model NEMO. The target resolution of 1km was reached. Analogously coupled 10km-resolving simulations based on ICON and EC-Earth were established and successfully run on some of the largest supercomputers in Europe; IFS could even be run on Summit, the fastest machine in the world. The results suggest that simulations at 1km are feasible; yet, there is still an enormous gap between achieved throughput in terms of simulated years per day and the target throughput rate of 1 simulated year per day, which is the minimal requirement for most of the relevant science cases. This calls for significant continued efforts in terms of mathematical, algorithmic, and software development.
Yet, the ESiWACE efforts have already created valuable synergies with the science case. In late 2017, the project DYAMOND was launched. DYAMOND stands for DYnamics of the Atmospheric general circulation Modeled On Non-hydrostatic Domains and targets the inter-comparison of international global models, running at highest affordable resolution. Both atmosphere-only models IFS and ICON, participated in DYAMOND, and ESiWACE organised two hackathons to bring together international leading experts in the field and to leverage synergies on DYAMOND data evaluation.
2) Usability: Handbooks, Trainings, Workflows, Software Management
To improve the overall workflows required by the weather and climate models, several developments were carried out in ESiWACE. The workflow engine Cylc was supported and improved to provide a robust and scalable solution meeting the community needs. Furthermore, data analytics workflows were advanced based on the analytics workflow manager Ophidia, including the extension of interfaces and capabilities for analytics workflows, HPC scheduling and parallel file system integration.
To prepare the community and supercomputer administration for weather and climate prediction at exascale, handbooks on the software stacks required by advanced weather and climate simulations were established. The program package manager SPACK was extended to support various software packages relevant to weather and climate modelling, which facilitates software installation procedures on varying supercomputing platforms and, thus, reduces system administration efforts.
Further, various workshops, trainings and user support were organised and supported through ESiWACE, including training sessions for the coupling software OASIS and the XIOS I/O software, user support for NEMO, EC-Earth and Cylc, and two HPC workshops.
3) Exploitability: Coping with the Data Avalanche
To understand the requirement for exascale storage systems and determine beneficial variants, various business models for storage systems and corresponding infrastructures were investigated. A major focus was put on the development of the Earth System Data Middleware (ESDM). This software is an interface between storage hardware and the data descriptions used by Earth system scientists. In particular ESDM enables the use of multiple storage backends in parallel – a promising achievement for handling future big data in weather and climate predictions.
The evaluation of the ESDM performance was conducted in large benchmark runs and demonstrated the capability to handle high-resolution data in future production runs.
A particular impact and success of the European centre of excellence ESiWACE is that it could gather and heavily foster synergies throughout Europe on the joint way towards exascale. This has constituted itself in the form of various meetings, joint code developments and trainings, We carry-on the idea in the project ESiWACE2 that has started in January 2019 and will continue driving weather and climate modelling towards exascale computing.