Cross-modal local structure consistency for drone-based weakly aligned visible-infrared object detection

Drone-based VISible-InfraRed (VIS-IR) object detection is valuable for civilian and military fields like intelligent transportation, reconnaissance and surveillance etc. However, sensor imaging differences and complex target movements often result in local spatial misalignment-induced weak alignment in cross-modal images, degrading detection accuracy. The core challenge is thus to achieve effective alignment and fusion of cross-modal features. Most mainstream methods rely on global arithmetic operations for alignment and adopt global cross-attention for fusion, ignoring inter-modal distribution differences, which aggregate irrelevant or conflicting features, cause feature confusion and dilute critical fine-grained local structures. Given that identical objects share similar geometric shapes and spatial contours, their local structural features are highly consistent. Motivated by this, this paper proposes a Local-Aware Cross-Modal Alignment and Fusion (LA-CMAF) method. In feature alignment, a Local Cross-Correlation Alignment (LCCA) module is proposed. It builds dense cross-modal feature associations in local neighborhoods, explicitly models fine-grained inter-modal offsets, and conducts layer-wise precise offset estimation. In feature fusion, a Local Attention Fusion (LAF) module is introduced, which restricts attention to local regions, enabling the model to concentrate on semantically consistent local cross-modal structures and minimizing interference from irrelevant distant regions. Additionally, a Patch Modal Normalization (PMN) module is designed to enhance the accuracy of cross-modal similarity scores using local statistical parameters. Experiments conducted on three public datasets—DroneVehicle, DVTOD, and VEDAI show that LA-CMAF achieves mAP scores of 83.6%, 88.3%, and 77.3%, respectively. Operating at a rapid 115 FPS with a lightweight architecture of only 4.7 × 10 6 parameters and 22.4 GFLOPs, the proposed method outperforms advanced detectors and demonstrates strong potential for resource-constrained, real-time applications.