Opening the floodgates for supercomputing fluid simulations
High-performance computing (HPC) enables physical processes to be accurately simulated, avoiding expensive and time-consuming real-world experiments. A key beneficiary is the field of computational fluid dynamics (CFD), where models predicting the transfer of heat and mass in fluids are used to design applications for sectors, ranging from aerospace to medicine. “Already a mature technology, future CFD growth depends on more effectively tapping HPC opportunities,” explains Fred Mendonça from ESI Group and principal investigator of the exaFOAM project. The project boosted the performance of OpenFOAM, a highly popular software for CFD workflows, enhancing its ability to leverage the power of HPC throughout the process chain. By increasing the understanding of areas such as solid mechanics, aerodynamics and heat transfer, OpenFOAM optimises the design of products for ground-vehicle engineering, aerospace, power generation and biomedical applications, amongst others. “Virtual engineering relies on combining HPC’s processing power with the versatility of artificial intelligence techniques. Our work, independent of any particular platform or chip, enables the immense calculations needed to train the algorithms underlying these cutting-edge techniques,” adds Mendonça.
Information bottlenecks
During time-consuming computer processing, users rely on receiving intermediate information to assess progress. This data exchange (I/O) can become a bottleneck, especially if the processing core stops until the data exchange is complete. The project used open-source software, such as ADIOS-2, to maintain efficient I/O file access while running on a large numbers of central processing unit (CPU) cores. As a result, OpenFOAM successfully ran on HLRS’s Hawk supercomputer, where it was deployed to 500 000 cores, for which the project received the HPC Technology Innovation Excellence award. “To the best of our knowledge, this is the largest number of cores ever deployed for an OpenFOAM simulation,” says Mendonça. Complementing these enhancements, in 2023, OpenFOAM’s release software included exaFOAM’s profiling tools, which identify processing bottlenecks.
Legacy code
The project also enhanced OpenFOAM’s code so it could run on CPU and graphics processing unit (GPU) new chip-architectures. The main challenge to overcome was that legacy CPU code (such as C++) doesn’t necessarily work on more powerful GPUs. The team developed a solution for one part of the CPU’s code – known as the linear solver – to be handed over from the CPU to GPU. Work also started on enabling legacy CPU code to be executed directly on GPUs, using industry-standard accelerators (OpenMP). While the I/O and GPU solutions resulted in increased performance boosts, the team also tweaked some of exaFOAM’s algorithms for more incremental gains and to reduce communication bottlenecks across the code. “Combined, these approaches upped OpenFOAM’s performance at least tenfold,” notes Mendonça.
Industry applications
The project launched several HPC performance benchmarks (Industrial applications and Grand Challenges) and made them publicly available. “We selected everyday examples, across various sectors, critical to things like: improved equipment design and performance, fuel efficiency for cost-effectiveness, ensuring public safety and comfort and reducing environmental impact,” explains Mendonça . Specific industry challenges included: reducing vehicle drag to increase range and optimising turbine placement in windfarm arrays to maximise energy generation. These resulted in 19 benchmark cases. The project team has now partnered with the EU-funded EXASIM project, which shares the objective of enabling exascale-capable CFD simulation using OpenFOAM.
Keywords
exaFOAM, EuroHPC JU , computational fluid dynamics, CFD, algorithm, code, I/O, GPU, CPU, chip, core, artificial intelligence, HPC