OpenGadget3 GPU solver tests

We present an in-depth evaluation of the scalability and accuracy of the GPU porting of the N-body code for hydrodynamic cosmological simulations. While technical details of our GPU porting were presented in Ragagnin et al. (2020), in this work we focus on assessing the accuracy of the ported modules: the short range gravity integrator, the different components of the hydrodynamic solver, and the conjugate gradient solver for thermal conduction. We ran several tests that gradually increase the number of physical modules included: a gravity-only cosmological simulation; a hydrodynamical shock tube test; a non-radiative zoom-in simulation of a galaxy cluster in a cosmological box; and a full-physics zoom-in simulation of a galaxy in a cosmological box. Comparing the results obtained with the GPU implementation to those from the classical CPU version, we find excellent agreement across all tests, with small differences on very small scales. For the individual physical modules, we find a GPU chip-to-chip speedup ranging from about3-5. For more complex cosmological and hydrodynamical setups, where a large number of physical processes and overheads contribute to the total workload, the observed total chip-to-chip speedup (with the same number of nodes and CPUs per node) is about2-3. We ran our tests on four different supercomputers: Leonardo Booster (CINECA), MareNostrum-V (BSC), SuperMUC-NG2 (LRZ), and the CIP cluster of the Faculty of Physics at the Ludwig-Maximilians-Universit"at (LMU).