Addressing Training Challenges in Federated Learning Using Adaptive Gradient Reshaping

Smartphones and mobile devices are becoming more and more popular, where aspects such as personalized recommendations and speech recognition have become essential. With the widespread adoption of mobile devices, federated learning (FL) has garnered significant attention as a distributed machine learning paradigm that safeguards data privacy. However, FL also faces a series of performance challenges, and the training process often suffers from model divergence or convergence that is too slow, slow parameter updates, and overfitting. We propose FedAvg with adaptive gradient reshaping (FedAVG-AGR) and FedSAM-AGR, which can alleviate the problems encountered during model training, such as the gradient vanishing and gradient explosion issues. Extensive experiments demonstrate that our proposed methods achieve superior accuracy and remarkable robustness compared to baseline algorithms across various attack scenarios and heterogeneous data distributions.