A Taxonomy of Clipping-Based Differential Privacy in Federated Learning

Federated Learning (FL) makes it possible to train models collaboratively over dispersed datasets while upholding strong privacy guarantees, most commonly via Differential Privacy (DP). The standard algorithm, Differentially Private Federated Averaging (DP-FedAvg), relies on per-client gradient clipping, a critical mechanism for bounding sensitivity to enable noise addition. However, the choice of clipping strategy is a complex problem that significantly impacts the trade-off between privacy and model utility. This paper presents a taxonomy of gradient clipping methods within DP-FL, categorizing them into three primary paradigms: static clipping, adaptive clipping, and gradient normalization. We analyze these approaches through the crucial lens of the underlying gradient distribution, with a focus on their performance under both classical light-tailed (e.g., sub-Gaussian) and empirically prevalent heavy-tailed assumptions. By examining the advantages and disadvantages of each method when applied to these distinct distributions, this work provides a lens for understanding and selecting appropriate clipping strategies in practical FL scenarios, ultimately clarifying the path toward more robust and effective private machine learning. In addition, we examine recent advances in addressing heavy-tailed noise and identify promising research directions to strengthen the robustness of DP-FL systems.