Prompt-Guided Unsupervised Joint Learning for Infrared and Visible Image Registration and Fusion

To address the challenges of fusion degradation introduced by registration-induced blurring, semantic loss from redundant representation during fusion preparation, and the inability to maintain dynamic consistency between registration and fusion in two-stage training, we propose a prompt-guided unsupervised one-stage framework for semantic registration and fusion, called UPRFNet. It integrates registration and fusion into a unified pipeline, consisting of two components: Unsupervised Euclidean Distance-Guided Multi-Scale Progressive Estimation (UESRNet) and Style-Prompt Guided Dual-Branch Attention with Semantic Interaction Fusion(SPDANet). UESRNet employs a multi-stage registration strategy combined with multi-scale Euclidean distance constraints to estimate deformation fields in a coarse-to-fine manner. The aligned semantics, reference semantics and prompt semantics are then directly fed into SPDANet, which performs semantic fusion and decoding to generate fused images, effectively avoiding information loss caused by repeated semantic representations. To evaluate the effectiveness and generalization of our method, we conduct three types of experiments: multi-modal registration and fusion to assess joint performance, multi-modal registration to validate UESRNet, and multimodal fusion to evaluate SPDANet. Experiments on multiple infrared–visible datasets demonstrate that the proposed framework effectively achieves accurate multimodal image registration and fusion.