Improving glioma grade classification with a hybrid CNN-transformer model

Purpose Accurate glioma grading is critical for treatment planning and prognosis. While convolutional neural networks (CNNs) capture fine-grained local features and transformers model long-range dependencies, each alone has limitations. This work investigates whether a hybrid CNN–Transformer architecture can improve classification performance on multi-sequence MRI. Approach We evaluated ten machine learning/deep learning models, including a radiomics approach, simple convolutional neural network (CNN), long-short term memory (LSTM), ensemble model, ResNet-based transfer learning, Vision Transformers (ViT, DeiT-base, DeiT-small, and DeiT-tiny), and a hybrid model that combines CNN features with DeiT Tiny. Performance was assessed using accuracy, precision, recall, and F1-score. Model interpretability was explored with Grad-CAM, attention maps, and t-SNE feature visualizations. Results Transformer-based models consistently outperformed CNNs and recurrent architectures, with DeiT Tiny achieving the best standalone performance (F1 = 0.90). The hybrid CNN+DeiT Tiny achieved the highest overall performance (F1 = 0.96, precision = 0.95, recall = 0.97), while maintaining practical efficiency (13 ms/slice inference, 89 MB model size). Interpretability analyses showed that the hybrid model effectively integrates local and global features, and t-SNE confirmed strong feature separability between glioma grades. Conclusions Combining CNNs and transformers yields superior accuracy, generalization, and interpretability for glioma grading, while remaining computationally feasible for real-time use. The hybrid model's accuracy and efficiency can help make glioma grading more reliable and useful in real clinical practice.