FedShieldLLM: Measurement, Detection and Protection of Privacy Leakage in Federated LLMs

Large Language Models (LLMs) exhibit remarkable language understanding and problem-solving capabilities but are heavily reliant on vast amounts of high-quality data, which raises significant privacy concerns. Federated Learning (FL) presents a decentralized training paradigm that minimizes raw data sharing but is still vulnerable to privacy attacks that exploit shared model updates. Federated LLMs, especially dialogue models, are particularly prone to query-based attacks, which can lead to the unintentional disclosure of memorized private information. This study performs extensive experiments across multiple model sizes and federated architectures to assess the extent of query-based privacy leakage and evaluate existing protection mechanisms. To enable standardized and reproducible privacy assessments, we propose a multi-stage privacy leakage detection method, integrating structured privacy attribute extraction, controlled injection, realistic prompting, and quantitative evaluation under both generalized and personalized privacy scenarios. Additionally, we introduce FedShieldLLM, a lightweight, adaptable client-side anonymization framework that performs personalized data de-identification prior to federated training, effectively mitigating privacy risks while preserving task utility. Our contributions include: (1) a systematic analysis of query-based privacy leakage in federated LLMs; (2) the design of a reproducible, architecture-agnostic multi-stage detection method; and (3) the development of FedShieldLLM for secure and flexible client participation in privacy-sensitive federated learning.