Trivance: Latency-Optimal AllReduce by Shortcutting Multiport Networks
Abstract
AllReduce is a fundamental collective communication operation in distributed computing and a key performance bottleneck for large-scale training and inference. Its completion time is determined by the number of communication steps, which dominate latency-sensitive workloads, and the communication distance affecting both latency- and bandwidth-bound regimes. Direct-connect topologies, such as Google's TPUv4 tori, are particularly prone to large communication distances due to limited bisection bandwidth.
Publication details
- Journal / source
- Proceedings of the ACM SIGCOMM 2026 Conference
- Publisher
- ACM
- Pages
- 160–176
- Publication date
- 11 August 2026