Joint Registration and Fusion Network for Dual-Frequency SAR Images Based on Semantic Information

Synthetic aperture radar (SAR) provides the high- resolution imaging capability and can obtain the ocean and land target information under complex observation conditions. Dual-frequency SAR images exhibit the complementary properties in the resolution and scattering characteristics, thus fusing dual- frequency SAR images can combine these advantages to provide the richer and more comprehensive information for downstream semantic segmentation tasks. However, existing fusion algorithms typically assume preregistered images and focus on the good visual effects while neglecting the semantic requirements of high-level vision tasks. To address this problem, a joint registration and fusion network for the dual-frequency SAR images based on the semantic information is proposed, which can achieve the high-precision cross-frequency registration and high-quality image fusion. First, in the image registration stage, the microwave vision data (MVD) semantic pseudo-color maps are introduced as the auxiliary information, and the semantic-consistency loss is used to guide the registration process, finally the fusion loss is fed back to the image registration as an optimization signal. Second, in the image fusion stage, the global spatial attention mechanism is used for the feature extraction and fusion, and then the semantic segmentation loss is incorporated to guide the image fusion so that the results can better conform to the high-level semantic structures. Experimental results demonstrate the effectiveness and correctness of the proposed method, which can outperform state-of-the-art approaches in the image registration accuracy, image fusion quality and downstream semantic segmentation performance.