CAN FEDERATED DIFFERENTIALLY PRIVATE MULTI-MODAL TRANSFORMERS IMPROVE RARE HEALTHCARE FRAUD DETECTION?

An ongoing challenge in healthcare insurance fraud detection is designing intelligent algorithms that identify subtle and evolving fraud patterns while preserving patient privacy. We propose FraudBERT-MM, a multi-modal framework that fuses structured claim features with unstructured textual explanations to form comprehensive claim representations. The architecture combines a BERT-based text encoder, an MLP-based structured encoder, a cross-modal fusion layer and a binary classification head. To address scarce labelled fraud examples, we apply prototype-based few-shot learning with episodic training. For privacy-preserving collaboration, models are trained in a federated learning setup augmented with Gaussian differential privacy, achieving a privacy budget of ε = 1.87. Unlike prior work that assumes large centralized labeled datasets and single-modality inputs, FraudBERT-MM integrates multi-modal learning, few-shot adaptation and formal privacy guarantees in one unified pipeline. We evaluate the approach on a real-world dataset of 4,000 claims with 83 features. FraudBERT-MM attains an F1 score of 0.882, outperforming centralized non-private, text-only and structured-only baselines. The model also converges 26% faster than the centralized baseline while preserving strong privacy, with only a small utility loss from noise. Experimental results indicate improved detection of rare and emerging fraud trends without compromising data privacy. We provide implementation details, ablation studies and deployment guidelines to support reproducibility and accelerate safe adoption in clinical and insurance settings across diverse operational contexts. These findings suggest that combining multi-modal representations, few-shot adaptation and federated differential privacy offers a practical and effective path toward deployable, privacy-aware healthcare fraud detection systems.