DistillVFL: A Knowledge Distillation-enhanced Vertical Federated Learning framework for scalable cross-silo collaboration

Vertical Federated Learning (VFL) is a privacy-preserving distributed learning paradigm where parties with disjoint features collaboratively train a unified Machine Learning (ML) model without leaking private data. This approach effectively addresses medical data fragmentation, allowing organizations to build unified models even when regulations like GDPR or HIPAA prevent raw data sharing. However, VFL faces a critical scalability challenge due to its static architecture. Onboarding new participants typically requires costly system-wide retraining, demanding that all original participants remain online. This bottleneck limits the growth of real-world collaborative ecosystems. To address this, we propose DistillVFL, a novel framework leveraging Knowledge Distillation for efficient client onboarding. Our approach employs a teacher–student paradigm where a pre-trained multi-party “teacher” model transfers knowledge to a new “student” client. This approach allows original clients to remain offline, eliminating re-participation requirements and additional computational costs. We evaluate DistillVFL through extensive experiments on four diverse medical datasets. The results demonstrate that student clients can achieve performance comparable to the comprehensive teacher model while drastically reducing computational overhead. DistillVFL provides a practical, scalable solution to the client onboarding problem, facilitating more dynamic and adaptable VFL collaborations.