Revisiting Communication Software Offloading for MPI+Threads: Reducing Contention and Improving Overlap on Many-Core Systems

Modern High-Performance Computing (HPC) systems are composed of multiple compute nodes featuring increasingly complex many-core architectures. Combining message passing (MPI) with multi-threading (MPI+Threads) is a promising programming model for achieving high performance on such systems, particularly by enabling the overlap of asynchronous communication and computation. Despite these advantages, MPI+Threads poses significant challenges and several techniques have been proposed for mitagtion, such as fine-grained locking, MPI endpoints, and partitioned communication introduced in recent MPI standards. In this work, we revisit the concept of communication software offloading and present libmmcso, an open-source library. The primary concept of libmmcso is to delegate communication requests to a dedicated thread which manages all MPI operations transparently. Using libmmcso requires no modification to existing MPI libraries or user applications.