Leveraging Modal Interaction and Window Dilation in Attention Network for Hyperspectral and Multispectral Remote Sensing Image Fusion

Hyperspectral and multispectral (HS-MS) image fusion aims to reconstruct high-resolution hyperspectral images (HR-HSIs) from low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI). Convolutional neural networks (CNN) have been extensively applied to this fusion task due to their powerful feature extraction capabilities. However, the limited receptive fields of CNN-based methods make it challenging to capture long-range dependencies in images, thereby restricting their performance in fusion tasks. Attention mechanisms, which can effectively capture global correlations in data, have emerged as a popular research topic. Despite this, existing attention-based hyperspectral fusion methods have not fully integrated and coordinated the two modalities, i.e., LR-HSI and HR-MSI, resulting in limited reconstruction quality. In this paper, we propose a Modalities Interaction and Window Dilation Attention Network for HS-MS image fusion, dubbed as MIAN. Specifically, our proposed MIAN introduces a modalities interaction attention block designed to explore, coordinate, and fuse the two modalities, facilitating information exchange and supplementation across different modalities. Additionally, we design a window dilation attention block that overcomes the local limitations of conventional window attention by establishing long-range connections across windows, thereby facilitating the handling of cross-scale self-similarity in remote sensing images. Furthermore, we incorporate channel attention blocks into each attention block to learn the channel dependencies which are the most important features for hyperspectral tasks. Extensive experiments demonstrate that our MIAN outperforms recent approaches on several hyperspectral datasets, achieving state-of-the-art performance.