Hyper_ALO_HybML: A Radiomics-Driven Ensemble Framework with Improved Ant Lion Optimization for Osteosarcoma Classification

This study presents Hyper_ALO_HybML, a hybrid machine learning framework enhanced through Improved Ant Lion Optimization (ALO) for accurate Osteosarcoma Classification from H&E-stained histopathological images. The radiomics-based texture, shape, and intensity features were processed with three ensemble classifiers, including Random Forest, XGBoost (eXtreme Gradient Boosting), and LightGBM, all tuned using the modified ALO to enhance feature selection ability, learning stability, and anti-overfitting. The dataset is composed of 1,144 images classified into Non-Tumor, Viable Tumor, and Necrotic Tumor. The experimental results demonstrate that the system with ALO-optimized XGBoost obtains the best performance, achieving an accuracy of 93.29%, a precision of 94.08%, a recall of 91.94%, and an MCC of 0.8947, respectively. LightGBM also holds similar generalization, with 92.71% accuracy, and Random Forest stops at 85.42%. Confusion matrix review reveals that gradient boosted models (optimized) deliver the most stable predictions among the classes. Results indicate that the integration of radiomics with ALO-guided hyperparameter optimization helps enhance Osteosarcoma detection and can aid in more reasonable histopathological decision-making. This hybrid approach offers a feasible route toward improved early diagnosis, minimizing errors in manual interpretation as well as enhancing computer-aided pathology.