Periodic Reporting for period 2 - Cosmo Plasmas (Cosmological simulations of radio bright plasmas) Reporting period: 2017-09-01 to 2018-08-31 Summary of the context and overall objectives of the project Merging galaxy clusters are among the most energetic events in the Universe. Their interaction generates shocks and turbulence in the ionised cluster medium, driving the amplification of magnetic fields and the acceleration of relativistic particles. This leads to the formation of large radio sources in clusters, whose origin is sparking a lively theoretical debate in the community. The ongoing leap in radio instrumentation, led by the European LOFAR observatory and by the JVLA, is revolutionizing the field by unveiling an increasingly complex picture. However we still lack numerical simulations to match the capabilities of these instruments. This roots in the complexity of modelling MHD, magnetic field dynamos, turbulence and cosmic-rays posing challenging demands to simulators.In this project, we aim to meet these challenges building a novel numerical platform that is based on the WOMBAT code. We worked in collaboration with experts of particle acceleration and Eulerian-MHD at the University of Minnesota and Ulsan National Institute of Science and Technology, and with the experts of high performance computing at Cray Inc. We developed a high-performance high-order MHD version of WOMBAT that shows unprecedented scalability and strongly increases the effective resolution and fidelity with respect to other existing numerical codes. The new platform is capable to overcome the limitations of the available MHD cosmological codes and algorithms thus allowing us to model the magnetic fields in galaxy clusters with unprecedented detail and fidelity. Large (PB-scale) simulations with our code will shortly provide the first steps to the eagerly awaited predictions for the next generation of cluster observations, revealing also an unprecedented view on the micro-physics in hot weakly-collisional plasmas. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far The project started in September 2015 with a 2 years out-going phase at the University of Minnesota. The main tasks for the initial part of the out-going phase was to take part in the development and testing of the existing MHD WOMBAT code. During this phase the researcher tackled truly novel numerical problems resulting in a strong benefit for his trainig and future career and for the value of the project. Results on the WOMBAT code were presented Mendygral et al (2017). In this paper we were able to show perfect strong scaling to 68 threads on Intel KNL many-core processors and industry leading weak scaling to 260.000 cores on the BlueWaters machine.During the out-going phase the researcher finalised also two publications connected with the Marie Curie project (Donnert et al 2016, 2017). A strategic decision that was taken during the out-going phase was to implement the high order WENO5 grid method into WOMBAT. Implementing a 5th order scheme to solve MHD equations instead of a TVD 2nd order scheme increases significantly the novelty of our numerical scheme and its capabilities to model complex magnetised flows as those expected in the cosmological framework. First steps in this direction were already taken at the end of the out-going phase, however the WENO5 implementation of WOMBAT was in fact the main task during the incoming phase.The incoming phase started on September 1st 2017 at the IRA-NAF.During the incoming phase the researcher successfully implemented and fully tested a 5th weighted essentially non-oscillatory scheme (WENO5) for Eulerian magneto-hydrodynamics in WOMBAT. The scheme doubles effective resolution with respect to state-of-the-art 2nd order schemes (PPM,TVD) and resolves instabilities even better and is much less susceptible to advection errors, which are a common problem in cosmological simulation with Eulerian codes (see FIGURE). First results on testing and comparison with other algorithms have been discussed in Donnert et al (2018), a longer paper is in advanced stage of preparation (Donnert et al to be submitted). Recently WOMBAT was scaled by the OpenMPI collaboration to 512.000 cores on the Trinity supercomputer at Los Alamos National Laboratory, USA. To our understanding our code now holds the record for the largest MHD simulation ever run.Having in hands a unprecedented highly scalable and high-precision MHD code, the last step of the work is to implement WENO5-WOMBAT in a full cosmological framework. The cosmological implementation of the WOMBAT code started at the end of the out-going phase and continued during the incoming phase. We successfully developed a 3D gravity implementation for Dark Matter in the code, final tests are ongoing and the results will be published soon (Nolting, Donnert et al, in prep). In these years the results of the project have been presented in several international conferences and workshops. The researcher established a website of the project at http://wombatcode.org , where he presents an introduction for the public, a blog for researchers and performance engineers, the public version of the code and all publications. The researcher organized a 1 week group workshop at the IRA-INAF in July 2018, where the whole collaboration met to discuss the future of the project. In conclusion during the 3 years of the project we have developed a innovative platform that is capable to overcome the limitations of the available MHD cosmological codes and algorithms thus allowing us to model the magnetic fields in galaxy clusters with unprecedented detail and fidelity. Large (PB-scale) simulations with our code will shortly provide the first steps to the eagerly awaited predictions for the next generation of cluster observations. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The initial phase of the program happened to be significantly slower than expected due to a number of unforeseen problems, but also because during the first year we decided to upgrade the MHD architecture of the WOMBAT code implementing a WENO5 MHD solver scheme. These unforeseen developments put the researcher in the favourable condition to tackle truly novel numerical problems. Indeed the work carried out during these 3 years generated a truly novel and powerful numerical platform. In fact the WENO5 implementation is the first high-order method in the field and by far the most efficient Eulerian MHD implementation currently available. DM gravity and on-the-fly load balancing capabilities of WOMBAT, based on asynchronous communication, are a completely novel approach to the long-standing problem of domain decomposition in the gravity-N-body problem at petaFLOP scale.The final goal of the return-phase was to put the researcher in the condition to run large numerical simulations using the cosmological WENO5-WOMBAT version of the code to study the evolution of the magnetic fields and turbulence in galaxy clusters. We basically achieved this goal and plan to run these simulations in the near future. Due to their innovative nature, these simulations will provide a ground-braking impact in our field and will allow us to explore the amplification and evolution of magnetic fields and turbulence in galaxy clusters and large-scale structure of the Universe.The advances in numerics in this project have established the researcher as a leading numerical scientist in non-thermal extragalactic astrophysics. Recently Donnert has led a review paper published in the Space Science Review Journal describing the state of the art of MHD turbulence simulations in clusters and the small-scale dynamo and the need of advanced MHD schemes like WENO5-WOMBAT. KH instability and magnetic energy tests
