Towards Feasible Clinical AI: Distilling Pathology Foundation Models with VIBE

The integration of large-scale Pathology Foundation Models (PFMs) into computational pathology has revolutionized histopathology by enabling robust performance on diverse diagnostic tasks using representations learned through self-supervised pretraining. However, the immense computational cost and memory requirements of these gigapixel-scale Vision Transformers (ViTs) render them impractical for routine deployment in clinical settings with limited hardware resources. Current compression strategies typically focus on creating smaller general encoder models or utilizing CNN architectures, approaches that either retain unnecessary capacity, need large retraining or fail to capture the rich semantic dependencies of transformers. To address this, we introduce VIBE (Visual Inference Embedding) Distillation, a geometry-preserving framework designed to transfer the global embedding structure of a foundation model into an ultra-lightweight, domain-specific student. By leveraging pointwise and geometric alignment based objectives like mean square error loss, we distill the 681M-parameter UNI-v2 model into a specialized 19M-parameter ViT-Small encoder tailored for ovarian carcinoma, circumventing the instability of intermediate feature matching. Validated on the CPTAC-OV and TCIA-OV cohorts, our best distilled student achieves a 35× reduction in parameters and 2.5x reduction in feature extraction time while retaining 97% of the teacher's performance in weakly supervised classification (ABMIL and linear probing) and retrieval tasks. We demonstrate that embedding space alignment effectively bridges the gap between research-scale capability and clinical feasibility.