An efficient differentially-private weighted support vector machine algorithm with noisy gradient descent

Abstract The proliferation of digital data has led to significant opportunities for analysis and strategic initiatives. Unfortunately, this has also raised privacy concerns. Differential privacy (DP) is a widely used framework for addressing such privacy issues. The DP guarantee can be achieved by incorporating randomness into the algorithm pipeline. However, generalized DP empirical risk minimization algorithms typically employ the output perturbation or objective perturbation, which do not perform well for high-dimensional data that are common in weighted support vector machine (wSVM) applications such as individualized treatment rules and outcome-weighted learning (OWL). To address these challenges, we propose a DP wSVM algorithm that incorporates gradient perturbation into the training process. Specifically, this approach injects noise at each gradient update, improving performance in high-dimensional and weighted settings while preserving strong DP. We evaluate the privacy-utility tradeoff of the proposed approach by comparing it with that of the existing DP-wSVM algorithm on three publicly available real datasets. We demonstrate that the proposed algorithm ensures accurate and efficient classification performance in pharmacogenetics and OWL applications, which benefit from our method despite their high-dimensional data, and maintains a more balanced tradeoff between privacy and utility. Additionally, a theoretical analysis of the privacy guarantee of the proposed algorithm has been performed, enables tight accounting of the overall privacy budget using Rényi DP. We argue that by clever algorithm design we can achieve excellent privacy-utility tradeoff for high-dimensional wSVM applications.