Periodic Reporting for period 2 - VECMA (Verified Exascale Computing for Multiscale Applications)
Reporting period: 2019-12-15 to 2021-12-14
VECMA uses computer simulations to predict weather and climate change, coronavirus pandemics, model refugees, understand materials, develop nuclear fusion, and inform medical decisions. But if we are to use simulations in order to affect real world problems then those simulations need to be reliable and trustworthy. In more technical terms, they need to be validated, verified, and their uncertainty needs to be quantified, so that they successfully model real life applications and be dependable decision-making tools.
VECMA has developed a software toolkit, called VECMAtk, to enable automated validation, verification, and uncertainty quantification (VVUQ) of computer simulations. While these tools are currently deployed on a number of in-house applications, they are applicable more widely and independent of scientific domain. More broadly, VECMA aims to create a unified European VVUQ package that computer simulations can be benchmarked against. To that end, VECMAtk has been made open-source and widely available in European high-performance computing (HPC) centres. Regular version updates have been done and released over the lifetime of this project (December 2021) by which time the goal was to create a legacy that will sustain itself beyond the end of the project.
Conclusion of the actions:
VECMA has achieved its overarching objective that was to enable a representative range of diverse multiscale applications across all science and engineering domains to run on current multi-petascale computers and future exascale environments, as demonstrated in the deliverable D4.3: Report on the implementation of non-intrusive and intrusive VVUQ techniques and D4.4: Report on application use cases. All Work Packages and their tasks were completed successfully. For the VVUQ Toolkit (VECMAtk), all three major annual releases of the VECMAtk in 2019, 2020 and 2021 and final release in January 2022 have been completed. The VECMAtk has be available after end of the project for a foreseeable future for the general public to access and use. VECMA has developed new UQP and VVP algorithms that were embedded in the Toolkit to support applications development. VECMA has implemented the VVUQ methods within a number of multiscale applications. The VECMA infrastructure team has built a HPC testbed that has supported extensively multiscale simulations to all partners.
VECMA has developed a software toolkit, called VECMAtk, to enable automated validation, verification, and uncertainty quantification (VVUQ) of computer simulations. While these tools are currently deployed on a number of in-house applications, they are applicable more widely and independent of scientific domain. More broadly, VECMA aims to create a unified European VVUQ package that computer simulations can be benchmarked against. To that end, VECMAtk has been made open-source and widely available in European high-performance computing (HPC) centres. Regular version updates have been done and released over the lifetime of this project (December 2021) by which time the goal was to create a legacy that will sustain itself beyond the end of the project.
Conclusion of the actions:
VECMA has achieved its overarching objective that was to enable a representative range of diverse multiscale applications across all science and engineering domains to run on current multi-petascale computers and future exascale environments, as demonstrated in the deliverable D4.3: Report on the implementation of non-intrusive and intrusive VVUQ techniques and D4.4: Report on application use cases. All Work Packages and their tasks were completed successfully. For the VVUQ Toolkit (VECMAtk), all three major annual releases of the VECMAtk in 2019, 2020 and 2021 and final release in January 2022 have been completed. The VECMAtk has be available after end of the project for a foreseeable future for the general public to access and use. VECMA has developed new UQP and VVP algorithms that were embedded in the Toolkit to support applications development. VECMA has implemented the VVUQ methods within a number of multiscale applications. The VECMA infrastructure team has built a HPC testbed that has supported extensively multiscale simulations to all partners.
Improved ensemble execution scalability and stability through new integrations between EasyVVUQ + FabSim3 and QCG-PilotJob has been achieved.
Extension and improvements of VECMAtk also included embedding of a wide range of automated procedures in FabSim3 and EasyVVUQ. These include template procedures for sensitivity analysis, automated validation infrastructure and many more.
There have been a lot of achievements made in the algorithms and formalisms work. We have developed the concept of Uncertainty Quantification Patterns (UQPs) and formalised new advanced algorithms to perform verification, validation and uncertainty quantification (VVUQ) for multiscale models, using novel methods for surrogate modelling and sensitivity analysis. Furthermore, we showed how surrogate models can be used to replace expensive single-scale model components, thereby speeding up computations and enabling UQ tasks that require many model evaluations. We also developed an approach to employ sensitivity analysis for model reduction of multiscale systems, thereby facilitating more efficient uncertainty estimation. The scalability of UQPs, and more generally of VECMAtk has been addressed at different levels, from theory to practice. We have developed various components in the VECMAtk that can handle large ensemble sizes, e.g. FabSim3 or the QCG tools.
For applications development, the different VECMA applications have explored a range of Uncertainty Quantification Patterns (UQPs) and Verification and Validation Patterns (VVPs) and demonstrated that these patterns apply across a large variety of fields. Monte-Carlo, Polynomial Chaos Expansion (PCE) and Stochastic Collocation (SC) have all been used to analyse the uncertainty in the Quantities of Interest (QoIs) driven by the uncertainties in the inputs or the inherent uncertainty of the process. Of particular interest was the use of adaptive methods which allows for UQ analysis to be applied for a much larger set of varying parameters.
VECMA has provided the HPC testbed where the QCG-PilotJob tool has also been intensively tested in regards to portability and scalability across multiple high-end computing resources. These HPC facilities have been provided to VECMA partners and successfully performed multiscale simulations and applications.
VECMA’s extensive dissemination activities, which were achieved and described in D6.5 cover a wide breadth of dissemination channels, materials, media, and target audiences.
Extension and improvements of VECMAtk also included embedding of a wide range of automated procedures in FabSim3 and EasyVVUQ. These include template procedures for sensitivity analysis, automated validation infrastructure and many more.
There have been a lot of achievements made in the algorithms and formalisms work. We have developed the concept of Uncertainty Quantification Patterns (UQPs) and formalised new advanced algorithms to perform verification, validation and uncertainty quantification (VVUQ) for multiscale models, using novel methods for surrogate modelling and sensitivity analysis. Furthermore, we showed how surrogate models can be used to replace expensive single-scale model components, thereby speeding up computations and enabling UQ tasks that require many model evaluations. We also developed an approach to employ sensitivity analysis for model reduction of multiscale systems, thereby facilitating more efficient uncertainty estimation. The scalability of UQPs, and more generally of VECMAtk has been addressed at different levels, from theory to practice. We have developed various components in the VECMAtk that can handle large ensemble sizes, e.g. FabSim3 or the QCG tools.
For applications development, the different VECMA applications have explored a range of Uncertainty Quantification Patterns (UQPs) and Verification and Validation Patterns (VVPs) and demonstrated that these patterns apply across a large variety of fields. Monte-Carlo, Polynomial Chaos Expansion (PCE) and Stochastic Collocation (SC) have all been used to analyse the uncertainty in the Quantities of Interest (QoIs) driven by the uncertainties in the inputs or the inherent uncertainty of the process. Of particular interest was the use of adaptive methods which allows for UQ analysis to be applied for a much larger set of varying parameters.
VECMA has provided the HPC testbed where the QCG-PilotJob tool has also been intensively tested in regards to portability and scalability across multiple high-end computing resources. These HPC facilities have been provided to VECMA partners and successfully performed multiscale simulations and applications.
VECMA’s extensive dissemination activities, which were achieved and described in D6.5 cover a wide breadth of dissemination channels, materials, media, and target audiences.
The VECMA VVUQ methods have been used in a number of cases that have advanced the state of the art in computational science. For example, accurate assessments of the uncertainties in protein-ligand free-energy calculations have been made possible through the use of VECMAtk. In the application domain of migration, considerable developments have been facilitated by VECMA in understanding better refugee dynamics and relating that to food security. In terms of more theoretical progress beyond the state of the art, one example is the project’s contribution within computer science to a method for queue wait-time prediction in supercomputing clusters. The method was designed for use as part of a multi-criteria brokering mechanism for resource selection in a multi-site HPC environment.
Having established the main UQPs, we have looked into developing performance models and testing those, as well as in developing more advanced UQPs. We have investigated formal methods for validation and verification of multiscale models. In terms of software implementation, we have developed the deep-track implementation of UQPs and VVPs, on optimising the scalability of VECMAtk, and on assembling combined VVUQ procedures and automation.
We expected this project to have tangible technical as well as societal impact, owing to its versatility and wide applicability. Upcoming exascale systems offer tremendous opportunities for computational science, however, important algorithmic and technological challenges still remain. It was these challenges that VECMA promised to overcome in order to be able to fully exploit these emerging opportunities and enable a paradigm shift to exascale computing. Through realisation of automated UQ and accelerated V&V by application developers worldwide, as well as through influencing next generation computer architectures, there will be improved fidelity of simulations irrespective of the application domain, leading to industrial and societal impact. Systematic dissemination, outreach and training associated with the releases of VECMAtk, have created impact by raising awareness on the case for high-fidelity exascale computing in multiple sectors.
Having established the main UQPs, we have looked into developing performance models and testing those, as well as in developing more advanced UQPs. We have investigated formal methods for validation and verification of multiscale models. In terms of software implementation, we have developed the deep-track implementation of UQPs and VVPs, on optimising the scalability of VECMAtk, and on assembling combined VVUQ procedures and automation.
We expected this project to have tangible technical as well as societal impact, owing to its versatility and wide applicability. Upcoming exascale systems offer tremendous opportunities for computational science, however, important algorithmic and technological challenges still remain. It was these challenges that VECMA promised to overcome in order to be able to fully exploit these emerging opportunities and enable a paradigm shift to exascale computing. Through realisation of automated UQ and accelerated V&V by application developers worldwide, as well as through influencing next generation computer architectures, there will be improved fidelity of simulations irrespective of the application domain, leading to industrial and societal impact. Systematic dissemination, outreach and training associated with the releases of VECMAtk, have created impact by raising awareness on the case for high-fidelity exascale computing in multiple sectors.