Federated Learning Privacy-Preserving Approach for Consumer-Centric Multimodal Healthcare Data

Electronic Health Records (EHR) data have become a valuable resource for biomedical research. By leveraging federated learning (FL), the potential of EHR data can be used without compromising privacy. However, most existing studies focus on privacy protection methods for device-centric distributed healthcare EHRs. The diverse modalities of EHR, such as consumer-centric clinical notes, lab results, and imaging data, limit the unimodal ability to capture complex relationships. FL models designed for unimodal data struggle to integrate these heterogeneous sources, resulting in suboptimal performance and increased risk of privacy breaches. This paper proposes a FL privacy protection method for multimodal data to address these limitations. The proposed method effectively handles diverse data types while ensuring robust privacy protection across the entire FL lifecycle. To prevent adversaries from stealing original data through reverse attacks, differential privacy perturbation is applied to the model parameters uploaded by each participant. Additionally, the Paillier cryptosystem is utilized for homomorphic encryption of local model parameters to protect local model information during training. Experimental results demonstrate that the proposed method provides comprehensive privacy protection for training data and models with minimal performance loss, making it suitable for federated learning scenarios involving multimodal EHR data.