SCMF-Net: Sparse Self-Attention Driven Cross-Modal Fusion for Robust Detection in Complex Road Scenes

This paper introduces SCMF-Net (Sparse Cross-Modal Fusion Network), a lightweight multimodal perception framework designed to enhance representation quality and inference efficiency while minimizing computational overhead. To address the sparsity and irregular distribution of LiDAR point clouds, an intensity-aware depth encoding strategy is proposed to enhance the structural cues in the depth modality. Additionally, a dual-branch backbone is employed to further strengthen feature extraction. Building upon this, FFLSA (Feature Fusion Local Self-Attention) is introduced to enable efficient cross-modal fusion. FFLSA leverages Self-Attention Clustering (SAC) to identify salient cross-modal regions, and Self-Attention Fusion and Purification (SAFP) to refine feature aggregation and reduce redundancy, forming an effective region-selection–refined fusion mechanism. Additionally, a Cross-Modal Feature Fusion Module (FFM) is proposed, which jointly models spatial and channel attention to enable adaptive RGB-depth interaction with fine-grained weighting. Extensive experiments on the KITTI dataset and a custom RGB–LiDAR benchmark under challenging conditions (fog, low light, and overexposed) validate the effectiveness of the proposed approach. SCMF-Net achieves 79.3% mAP on our dataset and 88.3% on KITTI, surpassing current state-of-the-art detection methods.