HFTM: A Hierarchical Feature Transformer Framework With Multiscale Consensus for SAR–Optical Image Registration

Synthetic Aperture Radar (SAR) and optical image matching is a critical prerequisite for achieving image fusion. Due to nonlinear radiometric differences and geometric distortions between SAR and optical images, existing methods struggle to extract homologous feature points, resulting in suboptimal registration accuracy. To address the mismatch issues in SAR-optical cross-modal image matching, a feature-based matching method named HFTM is proposed in this letter. Firstly, a hierarchical feature extraction network is applied on multimodal image pairs to extract global and local context information to obtain robust feature points. Consistent feature matching is then achieved in the local region through a Transformer-based attention mechanism and NeighConsensus (TNC) Matching network. The results of local matching are fed into the proposed global Position-encoded Probabilistic (P2) Matching network, which employs a detailed matching layer to generate an enhanced feature map, and predicts robust matching point pairs by a spatial expectation operator. Experimental results demonstrate the superior performance of our method, achieving a single responsiveness estimation accuracy of 89.43% on QXS-SAROPT and 86.21% on SEN12MS at a 5-pixel threshold.