Melan-Dx: a knowledge-enhanced vision-language framework improves differential diagnosis of melanocytic neoplasm pathology

Melanoma is one of the top 5 cancer types, causes most deaths among skin cancers, and can be frequently misdiagnosed. Recent pathology image foundation models remain difficult to make accurate differential diagnosis across over forty melanocytic neoplasm histologic subtypes. Motivated by the diagnostic reasoning process of dermatopathologists, we curated a high-quality image and knowledge corpus database containing 2893 images and 1102 knowledge entries annotated by expert dermatopathologists at the University of Pennsylvania. Leveraging this multi-modal dataset, we present "Melan-Dx", a knowledge-enhanced AI framework that augments frozen pathology vision-language models through retrieval from a curated vision-knowledge database, improving differential diagnosis at both patch and whole-slide levels. Melan-Dx, at its best performance, demonstrates 0.869 accuracy for binary classification, 0.699 Top-1 accuracy among forty-class classification, 0.915 ROC AUC for few-shot WSI tasks, and 0.925 AUPRC for fully supervised WSI tasks. Across all experimental settings, Melan-Dx shows improvements up to 13.8% over linear and fully finetuned methods, 23-70.6% over zero-shot approaches and up to 8.4% improvements in whole slide image classification. These findings suggest that a query database with a knowledge-enhanced AI framework can further improve existing pathology foundation models without fine-tuning the vision backbone. The code is publicly available at https://www.github.com/zhihuanglab/Melan-Dx-code .