Exact Linear Scaling of f-Divergences under Subsampling: Jensen-Shannon Divergence for Privacy Amplification
Abstract
We establish that all f-divergences satisfying f(1) = 0 exhibit exact linear scaling under subsampling: D_f(P || Q) = p · D_f(W₀ || W₁), where p is the sampling probability. This fundamental result holds for both Poisson sampling and sampling without replacement, providing a universal formula independent of the specific subsampling scheme. Key contributions: - General theorem proving linear scaling for all f-divergences (JSD, χ², KL, TV) - Extension to sampling without replacement with identical formula - Application to DP-SGD privacy accounting with concrete numerical examples - Comparison with f-DP (Gaussian DP) framework of Dong, Roth, and Su The Jensen-Shannon divergence satisfies JSD_sub = p · JSD_base exactly, contrasting sharply with Rényi differential privacy where subsampling amplification is inherently nonlinear (ratios exceeding 3× for small p). This positions JSD as a computationally simple tool for privacy amplification analysis. This work complements our shuffle model results (DOI: 10.5281/zenodo.18124012), establishing JSD as a principled framework for privacy amplification with exact analytical formulas.
Fields of Study & Subjects
Privacy-Preserving Technologies in Data
Random Matrices and Applications
Statistical Mechanics and Entropy
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 3 January 2026
- Language
- EN
- Document type
- Preprint