Deep Learning Based Classification of Bone Fracture: A Vision Transformer-CNN Approach

Accurate and effective classification of fractured and unfractured bones in X-Ray images is essential for rapid diagnosis and treatment planning. Bone fracture in joints requires precise classification which the traditional models struggle to classify due to the complexity in the joint structures. To identify bone fractures in joints, this study presents a hybrid model that is the conjunction of Vision Transformers (ViTs) with Convolutional Neural Networks (CNNs). The program correctly detects and isolates fractures across varied bone joints locations by using a comprehensive dataset of 9,340 X-ray pictures from various joints in the upper extremities. The initial dataset is combined and is divided with a 60:20:20 split: 60% training (5,599 images), 20% validation (1,866 images), and 20% testing (1,875 images). Our suggested architecture employs a pretrained ViT model to extract global contextual characteristics, with a CNNbased classifier fine-tuned to catch detailed local patterns. The hybrid design enables strong feature learning, resulting in higher classification accuracy. The model uses binary cross-entropy loss, optimized with the Adam optimizer, and is evaluated on previously unseen test data. The results of this study indicate that the hybrid ViT-CNN model is effective where it shows 98.79% as the training accuracy, 98.72% as the validation accuracy and 98.88% as the testing accuracy in classifying between the fractured and non-fractured X-Ray Classes. This study demonstrates the combination of the global and local feature extraction processes in leveraging the reliability and scalability of fracture detection in radiology.