DAMNet: RGB–LiDAR Fusion for Traffic and Water-Surface Detection Using Dynamic Direction Multiscale Scanning Priors and Deformable Window Cross-Modal

In traffic and water-surface environments, the fusion of RGB imagery and LiDAR point-cloud data can effectively improve detection performance. However, a key challenge arises when detecting target regions that occupy only a small portion of the image and point cloud, yet these regions are often complex. RGB provides dense appearance features but lacks reliability under reflective surfaces, low-texture environments, and varying lighting conditions; while LiDAR-based depth information, though offering more stable geometric features, suffers from data sparsity. To address these issues, this paper proposes the DAMNet (Deformable Attention Multi-modal Network) model, which utilizes a dual-stream backbone for extracting and fusing RGB and point cloud-derived depth maps. Through a progressive fusion strategy, DAMNet integrates features from both modalities across multiple spatial scales, improving object detection accuracy and robustness in marine environments. The three key modules are: LDMSSM (Learnable Direction Multi-Scale State Space Model) is designed as a direction-aware multi-scale state space module to enhance efficiency and alleviate computational overhead while maintaining accuracy; MASCAM (Multi-scale Asymmetric Spatial Channel Attention Module), which enhances cross-modal feature expression through multi-scale convolutions and asymmetric spatial pyramid structures; and DAMFM (Deformable Adaptive Masked Fusion Module), which ensures precise alignment, fusion, and refinement of features through deformable attention and hybrid dynamic masking strategies. Experimental results on the KITTI dataset demonstrate that DAMNet achieves state-of-the-art performance with mAP50 of 93.0% and mAP50-95 of 70.9%, using 30.65M parameters and 36.98 GFLOPs. The superior performance on water surface object detection tasks with mAP50-95 of 93.5% validates the effectiveness of the proposed multi-modal fusion approach in complex marine environments.