Micro-Benchmarking Communications Libraries on the MI300A Compute Partitioning Modes

Emerging APU-based supercomputers such as El Capitan rely on AMD’s MI300A, which uniquely supports GPU partitioning—exposing its six compute dies as one, three, or six logical GPUs. This capability enables finer-grained resource control but introduces new communication hierarchies whose performance implications remain poorly understood. This paper presents the first comprehensive characterization of MI300A GPU partitioning using the CommBench framework on a multi-APU El Dorado system. We evaluate peer-to-peer bandwidth and latency across SPX, TPX, and CPX modes and three communication libraries (MPI, RCCL, and IPC). Partitioning transforms the flat intra-node topology into a hierarchical one, with intra-package links reaching up to 587 GB/s—over 6× higher than cross-package bandwidth—while maintaining uniform low-latency communication. Beyond benchmarking, these results provide actionable guidance for runtime and collective library design on future heterogeneous systems, emphasizing adaptive SDMA utilization, and partition-aware communication to fully exploit the MI300A’s fabric.