Interpretable vision mamba for SAR images classification

Deep learning techniques have been successfully applied to object classification in Synthetic Aperture Radar (SAR) images, achieving remarkable performance. However, the current Transformer architecture still faces two limitations in SAR object classification: (1) Transformers occupies substantial GPU memory during training, which hinders large-scale model deployment and hyperparameter optimization. (2) the vectorization of image patches destroys spatial information among pixels, thus the represented features are semantic-agnostic. In this paper, we propose a lightweight and interpretable vision mamba (IVim) whose feature maps from deep Mamba blocks are visually understandable. IVim consists of two modules: Token Semantic Re-allocation (TSR) and Token Attention Selection (TSA). TSR re-allocates the semantics to feature maps by optimizing a novel loss function for an extra convolutional layer. TAS selects the most discriminative attention map by evaluating the quantity of semantics in attention matrices of different Mamba layer. Finally, we provide a coupling strategy to form a saliency heatmap to visually show the interpretability of IVim by merging feature map and attention map. Experimental results demonstrate that IVim can re-allocate semantics to deep features and achieve better classification performance compared to its counterparts by reducing 67.8% parameters.