TinyViT: a Lightweight Vision Transformer for Accurate and Interpretable Brain Tumor Classification From MRI

The magnetic resonance imaging (MRI)-based classification of brain tumors is important in early diagnosis and treatment planning. In spite of that, current types of deep learning, such as convolutional neural networks (CNNs) and transformers, have some limitations, including imbalanced representation of classes, small differences between classes, high computational costs, and low interpretability, that make them clinically inapplicable. In order to address these constraints, we introduce a fast Tiny Vision Transformer, TinyViT is a lightweight hybrid model that combines convolutional feature extraction with transformer-based global context modeling to achieve fast and highly accurate brain tumor classification. The model has been trained and tested on the PMRAM dataset which contains four classes: glioma, meningioma, pituitary tumor, and normal. As the results of the experiments show, TinyViT is able to perform at the state of the art level featuring overall accuracy of 0.9971, F1-score of 0.9970, Matthews Correlation Coefficient (MCC) of 0.9961, and both PR-AUC and ROC-AUC values are 1.000. Moreover, inference times are minimal (train 5.46 s and 1.27 s per test batch) and show a good trade-off between computation and error rates. Grad-CAM images indicate regions impacted by tumors in agreement with expert radiological evaluation and improve interpretability and reliability. The results indicate that TinyViT is an accurate, computationally inexpensive, and clinically interpretable method to automatically classify brain tumors, and has high prospects of being clinically applied in medical image processing.