Rankmap optimization for large scale HPC applications with simulated annealing based on MPI trace information

We propose a rank mapping optimization tool based on simulated annealing using Message Passing Interface (MPI) trace information. In distributed parallel executions of typical high-performance computing (HPC) applications, such as unstructured-grid simulations, MPI processes responsible for adjacent subdomains are often mapped to physically distant nodes, resulting in increased communication latency due to network congestion. This issue is particularly pronounced on large and directly connected networks such as the supercomputer Fugaku. To mitigate these communication issues, our method optimizes the mapping of MPI ranks to physical node coordinates by minimizing an evaluation function that represents communication costs derived from MPI trace logs. We employ the finite element simulation software FrontFlow/blue and the hierarchically structured finite volume simulation software CUBE to validate the effec t of communication time reduction. The proposed approach was tested on the supercomputer Fugaku with up to 3,072 nodes (12,288 processes), achieving approximately 48% reduction in communication time in the best case. While implemented with network topology of the supercomputer Fugaku in mind, the proposed method is applicable not only to directly connected networks but also to large-scale multi-hop systems with complex network topologies such as FugakuNEXT.