Skip to main content

Cosmological simulations of radio bright plasmas

Periodic Reporting for period 1 - Cosmo Plasmas (Cosmological simulations of radio bright plasmas)

Reporting period: 2015-09-01 to 2017-08-31

Merging galaxy clusters are among the most energetic events in the Universe. Their interaction generates shocks and turbulence in the ionised cluster medium, driving the amplification of magnetic fields and the acceleration of relativistic particles. This leads to the formation of large radio sources in clusters, whose origin is sparking a lively theoretical debate in the community.
The ongoing leap in radio instrumentation, led by the European LOFAR observatory and by the JVLA, is revolutionizing the field by unveiling an increasingly complex picture.
However we still lack numerical simulations to match the capabilities of these instruments. This roots in the complexity of modeling MHD, magnetic field dynamos, turbulence and cosmic-rays posing challenging demands to simulators.
In this project, we aim to meet these challenges and build a novel numerical platform that is based on the development of a high-performance MHD cosmological version of the numerical code WOMBAT. The new platform will be capable to overcome the limitations of the available MHD codes and alhorithms thus allowing us to model the magnetic fields and relativistic particles in galaxy clusters with unprecedented detail and fidelity.
This project will not only provide the first steps to the eagerly awaited predictions for the next generation of cluster observations, but reveal an unprecedented view on the micro-physics in hot weakly-collisional plasmas. We will achieve our ambitious goals in collaboration with experts of particle acceleration and Eulerian-MHD at the University of Minnesota and Ulsan National Institute of Science and Technology, and with the experts of high performance computing at Cray Inc. The advances in numerics in this project will establish the researcher as a leading numerical scientist in non-thermal extragalactic astrophysics and a key figure to interpret the radio data on the way to the SKA and the next generation of high-energy observatories.
The project started in September 2015 with the out-going phase. During the outgoing phase, in the first 2 years of the project, the goal was to give the opportunity to the researcher to be intensively trained in grid simulation codes and in the numerical implementation of MHD and of cosmic ray physics in these numerical schemes.
The main tasks for the initial part of the out-going phase was to take part in the development and testing of the existing MHD WOMBAT code. Such a initial phase happened to be significantly slower than expected due to a number of unforeseen problems. These unforeseen complications -however- put the researcher in the condition to tackle truly novel numerical problems resulting in a strong benefit for his trainig and future career and for the value of the project. Fixing the MHD implementation of WOMBAT and testing it took nearly 6 months and a paper was finally published in the beginning of 2017 (Mandygral et al 2017). In this paper we were able to show perfect strong scaling to 68 threads on Intel KNL many-core processors and industry leading weak scaling to 260.000 cores on the BlueWaters machine (FIGURE).
During the first year, Donnert finalised also two publications connected with the Marie Curie project (Donnert et al 2016, 2017). In particular in Donnert et al. 2016 it was proposed a model for the radio relic in the galaxy cluster CIZA J2242.8+5301. In this model the evolution of the relativistic particles in the shock-downstream region was obtained considering the effect of all the relevant mechanisms of particles energy losses and gains and a full exploration of the parameter space. Some of the mechanisms considered in this paper where previously unexplored leading to a more general theoretical framework that we plan to implement in WOMBAT in the near future.
A strategic decision that was taken at the end of the first year was to implement the high order WENO5 grid method into WOMBAT. Implementing a 5th order scheme to solve MHD equations instead of a TVD 2nd order scheme increases significantly the novelty of our numerical scheme and its capabilities to model complex magnetised flows as those expected in the cosmological framework. The WENO5 algorithm itself is remarkably simple and elegant, mostly because the underlying 4th order Runge-Kutta time integrator does not require tricks like an unsplitting to reach good accuracy. Preliminary tests of the algorithm indeed show excellent resolution and growth of Kelvin-Helmholtz instabilities close to the grid scale. This is a crucial test for our application, because these instablilties seed turbulence in the ICM.
The cosmological implementation of the WOMBAT code was the fundamental task planned in the second year of the out-going phase. This task has been delayed with respect to the original plans due to the unforeseen complications and the upgrade to the 5th order scheme described above. We started the cosmological implementation of WOMBAT at the end of the second year, testing of the 3D gravity implementation in the code is in progress.
As already explained the initial phase of the program happened to be significantly slower than expected due to a number of unforeseen problems, but also because during the first year we decided to upgrade the MHD architecture of the WOMBAT code implementing a WENO5 MHD solver scheme. These unforeseen developments put Donnert in the favourable condition to tackle truly novel numerical problems.
On the other hand, they caused an overall delay of the project, of about 6-9 months with respect to the original working plan. Consequently the astrophysical goals of the return-phase had to be scaled down, and for this reason we do not expect to focus too much in our program on the modeling of cosmic rays in the simulated clusters.
The final goal of the return-phase is to run large numerical simulations using the cosmological WENO5-WOMBAT version of the code to study the evolution of the magnetic fields and turbulence in galaxy clusters. Due to their innovative nature, these simulations will provide –by far- the best numerical platform in our field and will allow us to explore the amplification and evolution of magnetic fields and turbulence in galaxy clusters. As a follow up program we will insert relativistic particles as passive tracers in the simulated clusters to model particle acceleration and non-thermal emission.
More specifically during the first half of the return phase, we expect to finalize and publish the WENO5 implementation in WOMBAT (Donnert et al in prep), and the DM gravity in WOMBAT (Nolting, Donnert et al. in prep.). Both will be ground-braking for the field. The WENO5 implementation will be the first high-order method in the field and by far the most efficient Eulerian MHD implementation currently available. DM gravity and on-the-fly load balancing capabilities of WOMBAT, based on asynchronous communication, are a completely novel approach to the long-standing problem of domain decomposition in the gravity-N-body problem at petaFLOP scale.
After completing these steps, we expect the first cosmological simulations to be conducted during the second half of the return phase, by mid-end 2018, where we will target the small scale dynamo in cosmological cluster simulations.
Weak scaling of the TVD-MHD WOMBAT code on the BlueWaters machine