Markless Boat Berth Detection for Autonomous Surface Ships Through Fusion of LiDAR and Camera

In recent years, autonomous docking technology for autonomous surface vessels (ASVs) has attracted increasing research interest, and boat berth detection plays a crucial role in this task. However, common berth detection methods for ASVs adopt specific marks, such as ArUco codes, to implement boat berth detection, which significantly limits their applicability in practical deployments, since most conventional wharfs do not contain such markers. To support tasks involving human-less return to dock and enhance the generalizability of berth detection in real-world wharf scenarios, this article proposes a novel markless berth detection framework based on high-level scene semantics, without relying on artificial markers. Our proposed detection framework, based on the fusion of camera and light detection and ranging (LiDAR), comprises two important components. First, a LiDAR-camera fusion point cloud semantic segmentation model is constructed to generate 3-D semantic point clouds. Second, a berth extraction algorithm is designed to process point cloud segmentation results and detect free berths for ASVs. In addition, to improve the performance of the point cloud semantic segmentation model for ASV scenes, we developed a LiDAR-assisted view transformer module and a LiDAR-camera fusion attention module. Moreover, in our collected point cloud segmentation data set for ASV wharf scenes, our proposed method outperforms other point cloud segmentation baselines. Moreover, we conducted real-world prototype experiments to validate the feasibility and robustness of our markless boat docking detection method. The experimental results demonstrate that our proposed berth detection method has a high potential to support applications involving the autonomous docking of ASVs.