FedSPP: Lightweight Domain Modeling via Federated Learning with Soft Prompting

In recent years, pre-trained language models (PLMs) have shown great potential in the field of device log analysis to effectively process and understand unstructured text data. However, in practical applications, these models face challenges such as high computational resource requirements, difficulty in guaranteeing data privacy protection, and insufficient adaptation to specialized domains. To address these issues, we propose the federated soft prompting (FedSPP) framework, which combines lightweight domain modeling with federated learning techniques. Specifically, the framework employs soft-prompting techniques for efficient domain knowledge migration and a federated learning architecture for cross-organizational collaboration under data privacy protection. We evaluate the performance of the approach on the public dataset IMDB and the private domain ship dataset. Experimental results show that FedSPP reduces the amount of model parameters to 1.11 × 108 , the single communication overhead to 7.2 × 106 parameters, and the training time to 2 hours while maintaining comparable analysis accuracy compared to existing methods. These results confirm the effectiveness of our approach in resource-constrained scenarios and provide a practical solution for device log analysis.