Federated Split Learning via Low-Rank Approximation: A Communication-Efficient Approach

Federated split learning (FSL) has rapidly emerged as a promising paradigm for enabling ubiquitous intelligence in next-generation networks. However, current FSL approaches incur significant communication overhead and diminished training efficiency due to the frequent transmission of high-dimensional smashed data and gradients between devices and the base station. To address these issues, we propose a low-rank approximation (LoRA)-based FSL scheme, referred to as low-rank FSL. We analyze the convergence performance of low-rank FSL by considering the influence of LoRA rank on non-convex loss functions. To minimize a weighted sum of overall training latency and energy consumption in resource-constrained wireless networks, we formulate a long-term optimization problem by jointly optimizing computing frequency, power allocation, decoding order, LoRA rank, and split layer selection. An iterative optimization algorithm is then developed to solve this problem with low computational complexity. Numerical results demonstrate that our low-rank FSL reduces communication overhead by at least 300% while maintaining high learning performance. Moreover, our optimization algorithm achieves a low weighted cost in terms of training latency and energy consumption.