CTSNet: Cross-Modal Token Selection Network for Hyperspectral and Multispectral Image Fusion

Read full text Summarise with VIRAT
Authors: Yi'ao Jia, Yurong Qian, Jun Long, Yuanxu Wang, Sen Luo, Dongbin Hu, Weijun Gong
Cited by: 0
References: 39

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The fusion of low-resolution hyperspectral images (LR-HSI) and high-spatial-resolution multispectral images (HR-MSI) aims to combine the advantages of both modalities to enhance both spatial and spectral resolution, thereby generating high-spatial-resolution hyperspectral images (HR-HSI). However, existing methods still struggle to balance global-local feature modeling and computational efficiency, and they face a core challenge: spectral distortion during the upsampling due to the lack of cross-modal guidance. To address these issues, this paper proposes a cross-modal token selection network, termed CTSNet. A novel cross-modal guided spatial implicit upsampling pyramid (SIUP) structure is introduced. Unlike conventional implicit neural representation (INR) methods, SIUP directly incorporates MSI features as conditional inputs during the local Multilayer Perceptron (MLP) prediction stage, providing precise spatial priors to the HSI upsampling process. This design enables early-stage and deep fusion of cross-modal information, effectively resolving the problems of spatial detail blurring and spectral distortion during upsampling. Secondly, a token selection Transformer block (TSTB) is proposed to collaboratively extract global-local spatial and spectral features through a parallel dual-branch structure. A token selection attention mechanism (TSAM) is further introduced to significantly reduce computational complexity by employing an adjustable token selection rate. Finally, we design a multi-scale hybrid fusion (MSHF) module to achieve deep feature reconstruction. Experiments on four public hyperspectral datasets demonstrate that CTSNet outperforms current state-of-the-art (SOTA) methods in both qualitative and quantitative evaluations.

Fields of Study & Subjects

Advanced Image Fusion Techniques Remote-Sensing Image Classification Satellite Image Processing and Photogrammetry

Publication details

Journal / source
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Volume
19
Pages
16257–16270
Publication date
1 January 2026
Language
EN