Cross-Modal Compensation and Dual Deformation Prediction Correction: Breaking Barriers in Unaligned Infrared-Visible Image Fusion

Unaligned infrared (IR)-visible (VIS) image fusion typically follows a two-stage paradigm: first registration, then fusion. However, this two-stage framework lacks simplicity and compactness compared to one-stage pattern, thereby increasing the complexity of model training and deployment. To overcome these obstacles, we propose a one-stage method for unaligned IR-VIS image fusion. Specifically, two pairs of aligned IR-VIS images are used to construct full-modal features without modality discrepancy, which serve as guidance for uni-modal feature extraction. With the designed Cross-Modal Information Compensation (CMIC), uni-modal image features are converted into full-modal features. In this way, not only the complementary information of IR-VIS images are preserved, but also the adverse impacts of modality differences on cross-modal feature alignment are effectively mitigated. Our method resolves the conflicting requirements of traditional cross-modal alignment and fusion methods, achieving simultaneous feature alignment and fusion within a one-stage scheme. Besides, a Dual Deformation Prediction and Correction (DDPC) mechanism is devised to enhance feature alignment accuracy. It leverages modality discrepancy-free features and gradient information to predict two deformation fields, which are then integrated to generate a refined deformation field for precise cross-modal feature alignment. As for effective aligned feature fusion, an Adaptive Perception Feature Fusion Network (APFFNet) is developed, which can adaptively perceive the importance of each fused feature, preventing excessive information loss during the fusion process. Compared with existing two-stage methods, the proposed approach achieves a superior balance between alignment and fusion. Additionally, our model is more compact and lightweight, promising more efficient training and deployment. Extensive experimental results validate the effectiveness of our proposed method.