Skip to main content

Visual Exploration and Sampling Toolkit for Extreme Computing

Periodic Reporting for period 1 - VESTEC (Visual Exploration and Sampling Toolkit for Extreme Computing)

Reporting period: 2018-09-01 to 2020-02-29

Summary of the context and overall objectives of the project
Natural disasters such as the continuous wildfires in Australia or the extreme floods, earthquakes, and hurricanes all over the world are increasing. At the same time global diseases such as the current Corona virus, Ebola and Zika taking place. The implication of such dramatic events extremely threatens human lives and the social economy. To reduce the impacts, governments and aid organizations need reliable information’s to decide for efficient counter-measures.

Due to continuous advances in high-performance-computing, enhanced opportunities to model and simulate such physical, social, or economic phenomena are available. The produced predictions can help to create greater control for such situations. By enhancing these simulations with sensor data ever more precise and reliable predictions can be generated. Additionally, multiple simulations can be gathered into ensembles to increase statistical validity. Nevertheless, the integration of such complex and interactive workflows for urgent decision making, exploiting national or international HPC resources, is poor. The main reason is that such systems are not designed for interactive and urgent computing scenarios.

The overall objective of VESTEC is to bring such computational models, data fusion concepts, and ensemble analysis into workflows for urgent decision making that can save lives and reduce economic loss. The main challenge is to support and enable integrated, interactive access to todays and future extreme computing environments. Therefore, VESTEC integrates novel approaches to heterogeneous data fusion, urgent computing, and suitable data analysis methods together with visualizations to present the right information in the right way at the right time to the decision makers.
Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far
The project aims at developing and integrating urgent decision making scenarios as an emerging use mode for high performance computing (HPC). In order to evaluate the progress and success of the VESTEC project, three pilot applications are addressed:
1. Probabilistic forest fire forecast and monitoring
2. Mosquito-borne diseases
3. Space weather forecasting

WP2 has the overall goal to design the VESTEC architecture as a fundamental basis to integrate the current and future use cases. To guarantee that this architecture is appropriate, the requirements for every use case and application were documented first. Second, the sensor data used to augment and optimize reliability of the simulations were grasped. Based on this information, an initial system architecture has been developed. Additionally, in-situ computing interfaces are integrated into first simulation codes.

One major challenge when dealing with extreme data is efficient analysis. Especially for urgent scenarios, it is necessary to provide the right information at the right time. Therefore, WP3 develops and integrates in-situ data analysis methods to reduce data sizes and highlight prominent features. A novel and progressive computation method for generating topological proxies were developed. On top of the generated proxies, a reduction approach based on computing barycenter’s between persistence diagrams were developed. This method provides a basis to analyze simulation ensembles.

Another important functionality for urgent decision making is visualization. This process presents results to the decision makers and supports to decide for efficient counter-measures. Therefore, WP4 provides interactive and explorative visualization tools and algorithms. To support heterogeneous visualization a design plan for the extension of our tools were developed. Interactive exploration approaches for the simulation outputs together with the topological proxies are already developed and integrated.

As mentioned earlier current HPC systems are not designed for urgent computing. This limitation mainly originates trough the usage of classical batch systems. Additionally, HPC systems are isolated and processing external data or executing co-simulations becomes challenging. The goal of WP5 is to support timely and complex urgent computing workflows to overcome these limitations. Therefore, a federator approach was implemented. This federator manages multiple national or international HPC system and reduces execution times drastically. Furthermore, interfaces for the processing of external data were implemented.

WP6 targets at integrating the methods from previous work packages to establish and execute our pilot applications. Furthermore, simulation codes are extended and optimized to deal with sensor data. Results will be evaluated and feed back to improve the system design. Initially, the development and execution plan for the forest fire use case has been specified. Additionally, statistical models to simulate mosquito-borne diseases were enhanced to exploit sensor data.

Another important aspect of VESTEC is to make outcomes available to the public resources and build communities interested in these topics. WP7 has the aim to reach these goals. The main achievements are the implementation of the project website, establishing a Twitter channel, organization of workshops at international conferences. Besides these activities many scientific publications are created and made available as open access whenever possible. Additionally, tutorials and hackathons were held.
Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)
By integrating different metrics of the underlying HPC systems, the VESTEC federator approach established on an upper application layer, has successfully been used to improve the interactivity for urgent computing scenarios. This approach improves the latency and system response time, the time until results will be available to end-users. Therefore, this approach also promotes other use cases in the field of urgent decision making.

A novel algorithm for progressive computation of Wasserstein Barycenter’s, for a given set of input persistence diagrams, has been published on the IEEE Visualization 2019. This topological algorithm has impact to many other application domains dealing with scalar data. It supports highlight prominent features quickly. Additionally, the method improves the analysis of simulation ensembles also used in varying application domains. On top of that algorithm a new clustering approach has been published.

Since natural disasters such as wildfires and global diseases are increasing, modelling and simulating such phenomena becomes more and more important to save human lives and prevent socio-economic loss. To advance disaster modelling, VESTEC published a novel fire spread model considering uncertainties in wildfire simulations. Additionally, a mathematical model to simulate mosquito-borne diseases such as Ebola has been developed. These simulation models might be adoptable to other kind of diseases.

It is expected that the results of VESTEC will continuously increase the already existing interest in urgent computing. We believe that other applications domains in these fields will benefit from the developments. Also standard bodies and other relevant research programs and frameworks will profit. For the mosquito-borne diseases use case, public health organizations such as the WHO, ECDC and national health institutes represent the end-users. These may take advantage of timely and updated estimates on new and existing epidemic threads. The derived suppression activities will be safer and both, strategies and tactics of crisis forces will be more advanced in time and space.
Logo of the VESTEC Project