Infrared and Visible Ship Image Fusion Based on Adaptive Cross-Modal Feature Interaction and Multiscale Frequency-Domain Transformation

Infrared and visible image fusion serves as a pivotal technique for all-weather ship surveillance in maritime security applications, which can compensate for the inherent deficiencies of single-modal imaging under low-illumination and complex sea-state conditions. Nevertheless, existing fusion algorithms confront three core bottlenecks in coastal ship scenarios: first, insufficient robust feature representation capability against complex sea-sky backgrounds and multi-scale ship targets; second, inefficient cross-modal feature interaction between infrared and visible modalities; third, weak scene adaptability arising from static weight allocation in multi-scale frequency-domain fusion. To address these issues, this paper proposes an infrared and visible ship image fusion algorithm (AMTFuse) based on adaptive cross-modal feature interaction and multi-scale frequency-domain transformation, and constructs the IVSHIP dataset dedicated to ship image fusion. AMTFuse adopts an end-to-end framework and consists of three core modules: firstly, a contrastive learning-based feature enhancement network (CLFE) is developed, which mitigates small-target feature missing and insufficient modal discriminability in complex maritime scenarios via multi-scale feature distillation paths, cross-modal contrastive learning, and enhanced spatial attention; secondly, an adaptive cross-modal feature interaction module (ACMF) is devised, which breaks through the limitations of static interaction and poor scene adaptability in existing cross-modal Transformers by virtue of modal-aware gating, multi-level feature alignment, and dynamic attention updating; finally, a multi-scale dynamic frequency-domain fusion module (MSDF) is established, which implements multi-scale frequency-domain decomposition and content-aware dynamic weight fusion at the deep feature level, remedying the defects of spatial-frequency decoupling and static weighting in conventional frequency-domain methods. Experimental results on the self-built IVSHIP dataset as well as public MSRS and TNO datasets demonstrate that AMTFuse significantly outperforms existing state-of-the-art fusion algorithms in both quantitative metrics and visual effects. Furthermore, this paper validates that the proposed algorithm can be applied to boost the performance of high-level computer vision tasks (i.e., object detection).