Federated Sparsity Algorithm Based on Single-Cycle Dynamic Linear Cyclic and Norm Constraint

Federated Learning (FL) is a distributed deep learning framework that does not require collecting raw data from clients and performs specific tasks through distributed devices. Due to the heterogeneity caused by different data distributions between clients, clients tend to shift towards the local optimal solution during local iterative training and generate specific local models. Merging specific local models may deviate from the global optimal solution. This phenomenon will hinder the performance of the global model. Parameter regularization solves the problem of offset of the local optimal solution by restricting the update direction of the local model. Research on efficient parameter tuning and sparse federated learning has found that it is not necessary for the client to update all parameters in the global model during each round of training.Therefore, in this work, we first designed a novel sparse federated learning method: Fedl1/2, which can alleviate the degradation of FL performance by only updating some parameters in each training round. Usingl1/2regularization to control the update direction of each client while avoiding unnecessary parameter updates. To our knowledge, our study is the first to introducel1/2regularization in the field of FL to correct the local training of clients in FL. At the same time, we also presented Fedl1/2-ElasticNet, which extends the elastic network ofl1/2regularization in the FL field. In addition, considering that existing regularization coefficient adjustment plans are mostly static and there is relatively little research on dynamics, we propose a single period dynamic linear cyclic plan for regularization coefficient adjustment (SLC) and combine it with the Fedl1/2-ElasticNet algorithm to propose the SLC-Fedl1/2-ElasticNet algorithm. The superiority of the proposed method is verified by comparing it with eight state-of-the-art baselines.