WHFNet: A Wavelet-Driven Heterogeneous Fusion Network for High-Frequency Enhanced Optical–SAR Remote Sensing Segmentation

Synergizing the visual richness of optical imagery with the structural robustness of SAR is pivotal for land cover classification. However, the inherent heterogeneity and distinct data distributions between these active and passive modalities often lead to ”negative transfer”, where valuable modality-specific information is mixed during fusion. To overcome these limitations, this paper proposes a wavelet-driven heterogeneous fusion network (WHFNet) for co-registered optical and SAR images. Unlike normally symmetric architectures, WHFNet adopts a decoupled heterogeneous encoding strategy, leveraging the distinct inductive biases of Transformers and CNNs to preserve independent semantic representations. To bridge the modality gap, we introduce a cross-modal interactor (CMI) grounded in the 2D discrete wavelet transform. This module explicitly injects high-frequency information into spatial features, enhancing the representation of details. Furthermore, a spatial-frequency fusion module (SFFM) is devised to dynamically calibrate the discrepancies between modalities via subtraction operation, while a structural consistency constraint promotes semantically aligned predictions across modalities. Extensive experiments on four benchmark datasets (Xi’an, Pohang, WHU-OPT-SAR, and PIE-RGB-SAR) demonstrate that WHFNet establishes new state-of-the-art performance. Notably, it achieves substantial accuracy gains, particularly improving mIoU by 1.25% on the cloudless WHU-OPT-SAR dataset and 0.92% on the high-precision Xi’an dataset. The code will be publicly available at https://github.com/XD-MG/WHFNet.