Joint Classification of Hyperspectral and LiDAR Data Based on Dynamic Sparse-Dense Transformer and Mamba

Hyperspectral image (HSI) provides rich spatial-spectral information, while light detection and ranging (LiDAR) data offers complementary elevation details. As a result, the fusion of HSI and LiDAR data to improve classification performance has emerged as a prominent research direction. Recently, Mamba has demonstrated significant advantages in long-range sequence modeling with linear computational complexity, whereas Transformer is particularly effective at capturing short-range dependencies. To fully harness the complementary strengths of both architectures, this paper introduces a joint classification model for HSI and LiDAR data namedDynamicSparse-DenseTransformer andMamba(DSDT-Mamba). Specifically, we design a Dynamic Sparse-Dense Transformer (DSDT) to extract intra-modal features from HSI and LiDAR data separately. The DSDT integrates Sparse Self-Attention (SSA) and Dense Self-Attention (DSA): SSA alleviates the negative impact of poor query-key matches on feature representation, while DSA preserves sufficient information flow throughout the network to promote discriminative feature learning. Furthermore, features from different modalities are concatenated and transformed into a long multimodal sequence, which is efficiently fused using Mamba to leverage its long-range dependency modeling capability. Experiments carried out on four datasets serve to confirm DSDT-Mamba, with results showing that DSDT-Mamba yields competitive or superior classification accuracy compared to other state-of-the-art approaches.