SEMF-Net: A Spatial–Spectral Edge-Enhancement-Based Multistage Fusion Network for Hyperspectral and Multispectral Image Fusion

To tackle the issue of high-resolution hyperspectral imaging systems unable to obtain spatially fine hyperspectral images, fusion techniques for hyperspectral images with low-resolution and multispectral images with high-resolution have been proposed and have seen rapid development. However, current fusion algorithms based on deep learning often neglect the enhancement of spatial and spectral edge information, failing to fully exploit the correlation between spatial and spectral information and lacking comprehensive extraction of global-local spatial features. To mitigate these limitations, we proposed a spatial-spectral edge-enhancement-based multi-stage fusion network (SEMF-Net) for hyperspectral and multispectral image fusion. In SEMF-Net, we proposed a spectral edge mirror enhancement (SEME) block and a spatial edge enhancement (SEE) block to recover and enhance the spatial and spectral edge information during image fusion. Secondly, to leverage the correlation of the spatial-spectral information, we constructed the partition Transformer (Ptnformer) to realize the long-distance dependence modeling of the spectral information subregionally, which reduces the interference of unfavorable information. Finally, we designed a dual spatial awareness (DSA) block, which can not only perform global-local perception of spatial information but also perform sub-regional perception at the same time, thus realizing more comprehensive spatial information extraction. The experimental results on four remote sensing datasets showed that SEMF-Net can alleviate the spatial and spectral edge errors in the image fusion process and obtain fusion results superior to those of the current state-of-the-art fusion algorithms. The related code is available at https://github.com/cvmdsp/SEMF-Net.