A general two-stage framework for truck bucket pose estimation via monocular vision

Abstract Accurately estimating the 6D pose of the truck bucket is a critical challenge in the field of remotely controlled construction machinery, with significant implications for improving operational safety and efficiency. To address perception deficiencies and low efficiency in remote manual operations, this study proposes a monocular camera-based two-stage 6D pose estimation method for the truck bucket. Unlike traditional 3D object detection methods that require labeled point cloud data for training, this approach only necessitates simple circular video capture around the truck bucket to construct a sparse Structure-from-Motion (SfM) model. Subsequently, an aggregation-attention layer is innovatively employed to fuse associated 2D features into 3D feature descriptors. By leveraging self-attention and cross-attention layers, the 2D features of the query image are matched with the 3D features of the point cloud, enabling real-time 6D pose estimation without task-specific training. Experiments conducted on real-world dump trucks demonstrate that the proposed method achieves both high accuracy and efficiency. Specifically, the method achieves a single-frame localization time of only 0.0297 s and a video processing speed of approximately 10 FPS for 1920 × 1080 resolution videos. Under the 20 cm − 20 deg threshold, the pose estimation success rate reaches 93.8%. Furthermore, the method demonstrates excellent generalization capabilities in unseen complex engineering scenarios. These results indicate that the proposed method provides significant technical support for advancing the intelligence and efficiency of remotely controlled construction machinery operations.