3D pose estimation method for tea buds based on keypoint localization and point cloud for robotic picking

Morphological variations in developing tea buds in unstructured tea garden environments pose significant challenges to robotic picking. Therefore, accurate sensing of the spatial pose information of tea buds is essential to enhance robotic efficiency. This study proposes a method based on keypoint localization to detect four key features of tea buds. Subsequently, by integrating depth and point cloud data, it estimates the 3D pose of tea buds, providing critical information for harvesting tea buds in various postures. The GCYP-tea Pose detection network is proposed to enhance feature interaction by integrating the GLMixBlock module with the Efficient RepGFPN architecture, thereby strengthening hierarchical information propagation in multi-task detection and ultimately improving detection accuracy. A keypoint extension method based on the growth steps has been developed. By expanding the effective influence area of keypoints to form keypoint clusters, the effective range of keypoint mapping is enhanced, thereby mitigating mapping failures caused by incomplete point clouds. By introducing parameters α BL and α PE , the 3D pose of tea buds is evaluated using a 3D keypoint spatial coordinate vector representation, while simultaneously characterizing the optimal picking direction and angular adjustment for the end-effector of the picking actuator. Experimental results demonstrate the effectiveness of this method in predicting both the spatial posture of tea buds and the optimal picking posture of the end effector. This improvement enhances the understanding of tea bud spatial characteristics and provides accurate 3D positional coordinates along with directional guidance for robotic harvesting, thereby ensuring precision in automated operations.