Advancing Differentially Private Deep Learning: Algorithmic Improvements and Performance Evaluation

In recent years, machine learning and deep learning technologies have been widely applied across various domains. However, the privacy concerns associated with these technologies have become increasingly prominent. In this context, privacy-preserving training of deep learning models has emerged as a significant area of research. Differentially private Stochastic Gradient Descent (DPSGD), a privacy-preserving Deep Learning (DL) framework based on Differential Privacy (DP), has been extensively adopted to protect the privacy of training data. Nevertheless, the application of DP often leads to a decline in model performance. To address this issue, this paper proposes two enhanced methods: Particle Swarm Optimization-Based DPSGD (PSO-DPSGD) and Advanced DPSGD with Clipping-Ratio-Based Selective Update (ADPSGD). These algorithms aim to improve the performance of differentially private deep learning models through adaptive gradient clipping and optimized training trajectory, respectively. Experimental results demonstrate that under the same privacy budget (ϵ), the proposed methods achieve significantly higher model accuracy compared to the standard DPSGD algorithm. Furthermore, this study systematically summarizes evaluation work on DPSGD and investigates whether different strategies preserve equivalent privacy protection under the same privacy budget. Empirical evidence from membership inference attacks further demonstrates the practical guarantee of DPSGD variants. This study provides effective solutions for balancing model performance and privacy guarantees in differentially private deep learning, offering valuable insights for privacy-sensitive applications in real-world scenarios.