ViT-FESEL: A Multi-Stage Vision Transformer-Based Feature Extraction and Selection Framework for Osteoporosis Classification from Knee X-Ray Images

Early and accurate detection of osteoporosis is clinically critical for reducing fracture risk and enabling appropriate treatment planning.In this study, a multi-stage, multi-branch framework, Vision Transformer-Based Feature Extraction and Selection (ViT-FESEL), is proposed that integrates Vision Transformer (ViT)-based representation learning with a systematic feature extraction and selection strategy for osteoporosis classification from knee X-ray images.The proposed method aims to refine the high-dimensional deep representations produced by ViT backbones and generate discriminative features that can be effectively utilized by both classical machine learning algorithms and ordinal learning strategies.Within the ViT-FESEL framework, Branch-A is designed for binary diagnosis, Branch-B for ordinal modeling of osteoporosis severity levels, and Branch-C for an ensemble structure that fuses multi-scale representations extracted from different ViT backbones.Experimental evaluations are conducted using a five-fold cross-validation scheme with various backbone architectures and patch sizes.The results demonstrate that the FESEL module significantly enhances inter-class separability and that the Branch-C ensemble structure achieves the highest and most stable performance with an accuracy of 89.77%.Although ordinal learning results exhibit lower absolute accuracy, they successfully capture the ordered relationships among classes in a clinically meaningful manner.Overall, this study presents a hybrid, modular, and generalizable solution for osteoporosis detection from knee X-ray images and highlights the proposed ViT-FESEL approach as a strong candidate for clinical decision-support systems.