Multiscale Classifier Ensemble Knowledge Distillation for Remote Sensing Image Classification

To address the problems of significant remote sensing image ground object scale differences and traditional classification models struggling to balance accuracy and lightweight performance, this paper proposes a multi-scale classifier ensemble knowledge distillation method (MCEKD) for remote sensing image classification. Based on deep neural networks (DNNs), the method constructs multi-scale classifiers as teacher models and transfers multi-scale ground object discrimination knowledge to corresponding student models through ensemble soft labels. Experimental results on the UC Merced dataset, AID dataset, and NWPU-RESISC45 dataset demonstrate that when compressing the parameter quantity of mainstream models such as GCSANet model, Vision Transformer (ViT) model, and AFIMNet model, the performance of student models remains close to that of teacher models. The effectiveness of this method has also been demonstrated through experiments on the PatternNet, EuroSAT, RSD46-WHU, and RSOD datasets. Results of visualization experiments verify that the proposed method can better focus on important features.