Exact Linear Scaling of f-Divergences under Subsampling: Jensen-Shannon Divergence for Privacy Amplification

Read full text
Authors: Alex B. Shvets
Cited by: 0

Abstract

We establish that all f-divergences satisfying f(1) = 0 exhibit exact linear scaling under subsampling: D_f(P || Q) = p · D_f(W₀ || W₁), where p is the sampling probability. This fundamental result holds for both Poisson sampling and sampling without replacement, providing a universal formula independent of the specific subsampling scheme. Key contributions: - General theorem proving linear scaling for all f-divergences (JSD, χ², KL, TV) - Extension to sampling without replacement with identical formula - Application to DP-SGD privacy accounting with concrete numerical examples - Comparison with f-DP (Gaussian DP) framework of Dong, Roth, and Su The Jensen-Shannon divergence satisfies JSD_sub = p · JSD_base exactly, contrasting sharply with Rényi differential privacy where subsampling amplification is inherently nonlinear (ratios exceeding 3× for small p). This positions JSD as a computationally simple tool for privacy amplification analysis. This work complements our shuffle model results (DOI: 10.5281/zenodo.18124012), establishing JSD as a principled framework for privacy amplification with exact analytical formulas.

Fields of Study & Subjects

Privacy-Preserving Technologies in Data Random Matrices and Applications Statistical Mechanics and Entropy

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
3 January 2026
Language
EN
Document type
Preprint