Efficient Freespace Detection in Off-Road Environments Using Lidar–Vision–Language Multimodal Fusion

The ability of intelligent robots and vehicles to detect freespace efficiently and accurately in off-road environments is critical for ensuring safety. However, most multimodal approaches fuse on only Light Detection and Ranging (Lidar) and visual camera inputs, often overlooking semantic scene descriptions from environmental language. These techniques also struggle to achieve both high accuracy and real-time performance in off-road scenarios. To address these gaps, we propose Efficient Lidar-Vision-Language (LVL), a multimodal fusion framework that integrates Lidar, visual cameras, and language-based semantic scene descriptions for off-road freespace detection. This study introduces the Bidirectional Mamba Horizontal Vertical module, which efficiently encodes LVL multimodal information. Based on this module, the Stepwise LVL Fusion and LVL Feature Aggregation modules fuse modalities and aggregate multi-scale features with high accuracy. Ultimately, the Post Temporal Encoder module fuses multi-frame temporal information and employs an MLP to regress the binary image of the freespace. Experimental results on the ORFD dataset demonstrate that the model outperforms all comparison models in both precision and real-time performance, achieving state-of-the-art performance. A series of ablation studies further validate the effectiveness and interpretability of each module. Finally, qualitative experiments on the ORFD test set and real-vehicle data further confirm the high robustness and generalizability of the model.