The project aimed at developing and integrating urgent decision-making scenarios as an emerging use mode for high-performance computing. In WP2, the VESTEC system has been designed as a fundamental basis to integrate the current and future use cases. One major challenge when dealing with extreme data is efficient analysis. Especially for urgent scenarios, it is necessary to provide the right information at the right time. Therefore, WP3 has developed and integrated in-situ data analysis methods to reduce data sizes and highlight prominent features. A novel and progressive computation method for generating topological proxies has been investigated. On top of the generated proxies, a reduction approach based on computing barycenters between persistence diagrams has been developed. The integrated feature-based statistical methods provide the basis for analysis of simulation ensembles.
Another important functionality for urgent decision-making is visualization. This post-processing presents results to the decision makers and supports to find the right decisions for efficient counter-measures. Therefore, WP4 provided interactive and explorative visualization tools and algorithms. The classical scientific data visualization has been extended by statistical data presentation based on persistence diagrams. These diagrams can be filtered interactively to extract important clusters of information. These are called topological proxies and presented as georeferenced graphical objects in the visualization for the decision makers. The meaning is different for the different use cases. The usefulness has been discussed with the domain experts.
Current HPC systems are not designed for urgent computing. This limitation mainly originates trough the usage of classical batch systems. Additionally, HPC systems are isolated and processing external data or executing co-simulations becomes challenging. The goal of WP5 has been to support timely and complex urgent computing workflows to overcome these limitations. Therefore, a federator approach was implemented. This federator is able to manage multiple national or international HPC system which helps to reduces considerably the waiting time before those jobs are schedule for execution. This included also the incorporation of interfaces for the processing of external data.
WP6 integrated the methods from previous work packages to establish and execute all pilot applications. Furthermore, simulation codes have been extended and optimized to deal with the in-situ co-processing. With that, important results for the decision makers can now be processed immediately on the supercomputing and stream to the crises centers for interactive analysis. For all our three use-case, we have developed story lines to guide the goal-driven development and execution. The requirements for the features to be extracted were pretty different but made use of the same workflow offered by the VESTEC system, which shows the generic approach and the capabilities to be extended for further crisis scenarios. In dedicated execution evaluation events, the use cases could proof the efficiency and use-case specify outcome for urgent decision support.
Another important aspect of VESTEC is to make outcomes available to the public resources and build communities interested in these topics. WP7 aimed to reach these goals. The main achievements have been the implementation of the project website, establishing a Twitter channel, organization of workshops at international conferences. Besides these activities many scientific publications have been created and made available as open access whenever possible. Additionally, tutorials and hackathons were held. Eventually, we could present all our findings and implementations on our final workshop, which had to be carried out as a virtual event because of the current pandemic situation. A concluding journal paper is currently in preparation which will present the contribution and produced result by all partner of the consortium.
