Dual-Branch Token Selection Fusion Network for Hyperspectral Image Classification With Few Labeled Samples

Currently available transformer models have achieved satisfactory results in hyperspectral image (HSI) classification. However, the existing transformer models face two key challenges: First, the collaborative modeling of spatial-spectral features remains inadequate, hindering the models' ability to effectively extract and fuse complementary information between spatial patterns and spectral responses. Second, in practical scenarios with extremely limited labeled samples, complex deep models are highly susceptible to overfitting due to the severe mismatch between their massive capacity and limited training data. Significantly constraining its generalization capability and robustness. Therefore, developing classification algorithms that can achieve an excellent performance even under conditions of small sample sizes holds significant research value and is of practical importance. To remedy these drawbacks, a Dual-Branch Token Selection Fusion Network (DBTSFN) for HSI classification, is proposed in this paper. Specifically, DBTSFN employs a multi-module collaborative mechanism through a spatial-spectral cascade strategy (SSCS) to perform deep mining of spatial and spectral features, respectively, within a dual-branch architecture. A Gate-Enhanced Token Selection (GETS) block fully models spatial-spectral complementary information. Finally, through the classification decision fusion (CDF) module, the overfitting issue of the model under small-sample conditions was effectively mitigated, significantly enhancing its generalization performance in scenarios with limited labeled data. Since each decision prediction may exhibit different overfitting tendencies, combining multiple predictions can mitigate overfitting to enhance classification accuracy and stability. The results show that the proposed method significantly outperforms state-of-the-art approaches on all considered datasets and demonstrates the effectiveness and superiority of the proposed model for hyperspectral image classification when few labeled samples are used.