Predicting central lymph node metastasis in papillary thyroid carcinoma using a fusion model of vision transformer and traditional radiomics based on dynamic dual-modality ultrasound

OBJECTIVE: This study aimed to develop a novel fusion model based on dynamic dual-modality with B-mode ultrasound and superb microvascular imaging (SMI), combining Vision Transformer (ViT) and radiomics features to predict central lymph node metastasis (CLNM) in thyroid cancer patients. METHOD: In this retrospective diagnostic study, 310 patients with pathologically confirmed papillary thyroid carcinoma from two hospitals were included. We trained ViT models for B-mode and SMI, then extracted ViT and radiomics features from their video images. Initially, Single-modality models were developed, including the B-mode radiomics model (BMUS_RAD) and the B-mode ViT model (BMUS_ViT). Subsequently, Dual-modality models were constructed, encompassing the Dual-modality radiomics model (DMU_RAD), the Dual-modality ViT model (DMU_ViT), and finally, the integrated model DMU_RAD_ViT, to enhance the prediction of CLNM. The performance of each model was compared, and SHAP was utilized for the visual interpretation of the novel fusion model. RESULTS: Among all the models, the fusion model DMU_RAD_ViT performed the best (AUC = 0.901, p < 0.05). At the same time, the dual-modality model DMU_RAD(AUC = 0.856) and DMU_ViT(AUC = 0.832) is also higher than the single-modal model BMUS_RAD (AUC = 0.837) and BMUS_ViT (AUC = 0.789), respectively. SHAP analysis revealed that 16 radiomics and ViT features from both modalities contributed to the DMU_RAD_ViT model. CONCLUSIONS: The Dual-modality fusion model, integrating both radiomics and ViT features, can be utilized to predict CLNM.