PMFNet: A PCA-Driven Mixture-of-Expert Fusion Network for Hyperspectral Pansharpening

Hyperspectral (HS) pansharpening aims to fuse low-resolution hyperspectral (LRHS) images with panchromatic (PAN) images to generate high-resolution hyperspectral (HRHS) images. Most HS pansharpening methods fail to account for the information interference and computational burden caused by the inherent band information redundancy in HS data. In addition, the convolution operations can cause interference from surrounding features on the central feature during feature integration, resulting in inaccurate fusion features in texture rich areas. To address these issues, a principal component analysis (PCA)-driven mixture-of-expert (MoE) fusion network (PMFNet) is proposed for the HS pansharpening task. First, a PCA-based channel selection module (PCSM) is constructed to reduce band information redundancy and computational load by selecting the most representative bands. Then, a MoE-based dynamic fusion module (MDFM) is developed to adaptively allocate weights to expert networks by designing a learnable gating mechanism, achieving adaptive modulation and fine-grained fusion of features at each spatial position. Finally, a multi-frequency enhancement fusion module (MEFM) is developed to reconstruct the final fusion result by enhancing spatial features in different directions within the wavelet domain, as well as enhancing full band spectral features in the input HS image. Numerous experiments conducted on two simulated datasets and one real dataset demonstrate that the proposed method outperforms several state-of-the-art methods in both objective and subjective aspects.