Optimally Leveraging Stale Updates in Federated Learning

Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, practical FL systems face challenges due to partial client participation and device heterogeneity, which introduce significant variance in training updates and hinder model convergence. Existing methods naïvely reuse stale updates from inactive clients to stabilize training without accounting for varying levels of staleness or optimizing the balance between fresh and stale updates to minimize training variance. To address these issues, we propose FedVarM, a novel method that optimally leverages stale updates by adjusting client aggregation coefficients to minimize training variance. Additionally, we introduce FedVarM-Approx, an efficient approximation without extra computation on the client-side to ensure its efficient deployment. Experiments demonstrate FedVarM achieves 17.4% higher accuracy compared to methods that naïvely reuse stable updates without optimal balancing.