Cervi-LLM for real time colposcopy lesion detection and interpretable diagnosis

Colposcopy, a key procedure in cervical cancer screening for patients with abnormal TCT or HPV test results, requires extensive training for accurate diagnosis, yet diagnostic performance varies significantly across regions. This study presents Cervi-LLM, a multimodal MoE framework designed to assist in colposcopic detection by providing precise lesion localization and stratified diagnoses. The dataset included 126 cervical cancer, 692 HSIL, 306 LSIL, and 999 normal cases. Cervi-LLM integrates three core components: 1) YOLOMed for real-time, multi-scale segmentation of three staining modalities (saline, acetic acid, iodine); 2) joint fine-tuning of a LLM on both image features and clinical text using LoRA and progressive unfreezing; and 3) a two-level MoE architecture that fuses rule-based and data-driven expert outputs via dynamic gating, calibrated with clinical rules to deliver interpretable diagnoses (Normal/LSIL/HSIL( +)). In segmentation, Cervi-LLM achieved PA of 94.51% and meanIoU of 78.23%, outperforming U-Net (91.83%, 72.76%) and Polyp-PVT (93.69%, 75.34%). For three-class classification, it reached an overall ACC of 91.52%, exceeding junior physicians (65.88%) and senior physicians (76.57%). In detecting HSIL( +), it achieved sensitivity of 95.96% and specificity of 94.17%, compared to 67.58% and 81.67% for the senior physicians. The system processes images at approximately 30 fps and delivers a final diagnosis in 0.30 ± 0.05 min. Cervi-LLM effectively overcomes conventional colposcopy limitations, offering an accurate, real-time intelligent tool for cervical lesion screening and biopsy guidance.