DMHA-Transformer With Joint Convolution and Global Attention Enhancement for Hyperspectral Image and LiDAR Classification

Hyperspectral image (HSI) categorization plays a pivotal role in various Earth observation applications. Single-modal HSI is constrained by insufficient spatial resolution, spectral redundancy, and noise contamination, making it inadequate for high-accuracy land-cover classification and complex scene interpretation; thus, complementary multi-source information is urgently needed. Existing CNN-based approaches fail to adequately exploit the global sequential nature of spectral features, while Vision Transformers and related architectures incorporate a randomly initialized external CLS tokens, whose inductive bias often results in limited generalization. Given these limitations, we therefore develop JGAM-DMHA-Net, a new network architecture dedicated to joint classification by HSI and LiDAR data. The framework employs an enhanced dual attention for efficaciously combining supplementary information from HSI with LiDAR data. A joint network incorporating hybrid 3D-CNN, 2D-CNN and global attention mechanism (J-NHGAM) preserves both channel-wise and spatial-wise information by simultaneously modeling the interactions across channel, height, and width dimensions. It dynamically reweights the importance of different locations or channels on a global scale, enabling more effective information exchange and representation learning. In the DMHA-Transformer block, depth-wise (DW) convolution not only excavates deeper-level features but also establishes global dependencies among features at disparate spatial locations by leveraging complementary information sources, thereby endowing the model with superior generalization capability. The experimental results redon three public datasets indicate that the employed approach outperforms several state-of-the-art methods, with improvements of 0.95 %, 0.09 %, and 2.89 % respectively. The results provide clear evidence validating the advantages of our model redin achieving efficient multi-source data fusion, offering a practical solution for complex scenarios.