Performance comparison of machine learning models for predicting tumor grade in soft-tissue sarcoma using radiomics

Objectives This study aims to compare the performance of different machine learning models in predicting tumor grading in soft tissue sarcoma (STS). Methods Semantic imaging features, mean apparent diffusion coefficient (ADC), and Radiomics features were extracted from 126 sarcomas. Random forest, k-nearest neighbors (kNN) and support vector machine (SVM) were trained with and without feature selection and/or oversampling. The models were trained using either only Radiomics features, a combination of Radiomics features and ADC, or a combination of Radiomics features, ADC and semantic imaging features. The models were evaluated using an internal validation set with 5-fold cross validation. Results All models demonstrated decent performance, with accuracy > 0.64, F1 score > 0.74 and ROC-AUC > 0.62 across all models. The best-performing model was the combination of random forest with Synthetic Minority Oversampling Technique (SMOTE) and Recursive Feature Elimination (RFE), trained on the combination of Radiomics features, ADC and semantic imaging features (accuracy = 0.83, F1 score=0.88, AUC=0.87). A significant difference was observed between models trained on the combination of Radiomics, ADC and semantic imaging features compared to Radiomics alone (p = 0.016), as well as between the random forest and kNN models (p = 0.002). However, the differences between random forest and SVM, and between kNN and SVM did not reach statistical significance. Conclusion In this study, machine learning models achieved moderate to good performance in predicting tumor grade in STS. A random forest model combined with SMOTE and RFE and trained on the combination of Radiomics, ADC and semantic imaging features showed the highest performance. Key points •In 126 patients with soft tissue sarcomas, machine learning models trained on multiparametric MRI features demonstrated moderate to good discriminatory performance (accuracy 0.64–0.83, AUC 0.62–0.87) under internal validation. •Models combining radiomics, ADC, and semantic imaging features achieved higher performance than single-feature-set approaches, with a random forest model incorporating recursive feature elimination and oversampling showing the best results in this cohort. •Models trained without oversampling showed markedly reduced specificity, underscoring the importance of data balancing in imbalanced clinical datasets.