Synchronized Dual-Ring: A Synergistic Algorithm for Bandwidth-Efficient Collective Communication Leveraging Die-to-Die Direct Interconnects

The increasing scale of large language models (LLMs) has placed unprecedented demands on the communication infrastructure for distributed training. The architectural evolution towards Chiplet designs in modern processors, exemplified by the Huawei Ascend 910C, has introduced a new layer of high-speed, on-package communication via Die-to-Die interconnects. This architectural shift opens up an underutilized optimization space that standard collective communication algorithms, often designed for flatter network topologies, are not well positioned to leverage. This paper addresses this challenge by introducing the Synchronized Dual-Ring (SDR) algorithm, a novel, topology-aware approach designed to synergistically leverage these hierarchical network resources. The core innovation of SDR is the decomposition of a conventional communication ring into two parallel and synchronized sub-rings. This dual-ring structure enables simultaneous data transfer over both traditional NPU-to-NPU interconnects and high-bandwidth Die-to-Die links, effectively balancing communication load and maximizing aggregate throughput. We present a detailed design and implementation of SDR for key collective operations, including {\AR}, {\RS}, and {\AG}. We validate the efficacy and scalability of SDR through a comprehensive evaluation using analytical modeling, micro-benchmarks on an Ascend 910C cluster, and end-to-end training simulations. Our results demonstrate significant performance gains. Micro-benchmarks show that SDR improves bandwidth by up to 19.09\% for {\RS} and 18.48\% for {\AR} compared to the highly optimized vendor-specific Huawei Collective Communication Library (HCCL). Furthermore, end-to-end simulations reveal that SDR accelerates training throughput by up to 38.69\% over the standard Ring algorithm. Crucially, it improves upon HCCL's native collectives by up to 4.78\% for future trillion-parameter workloads. Our findings confirm that SDR provides a practical and scalable solution to mitigate communication bottlenecks, unlocking the full potential of modern hardware architectures for large-scale AI training.