Self-Supervised Learning with LiDAR-Camera Fusion for Construction Site Traversability Estimation

Navigating autonomous robots through construction sites requires accurate traversability estimation. However, there is a lack of labeled training data for construction site navigation, limiting the use of purely supervised learning methods. In addition, traversability estimation algorithms have to account for sensor limitations, such as camera images lacking depth information and LiDAR scans lacking color and texture information. To overcome these challenges, we investigate self-supervised learning approaches that combine 2D images and 3D LiDAR point clouds for ground robot traversability estimation in construction sites. We examine both early fusion, where raw RGB values from images and geometric features from point clouds are merged before being fed into a neural network, and late fusion, where embeddings from a segment anything model (SAM) encoder are projected onto 3D. Experimental results using the RELLIS-3D and ConSLAM datasets indicate that both early fusion and late fusion improved performance compared to the LiDAR-only method.