Two-stage progressive screw hole localization method for unstructured screw assembly

Robotic autonomous assembly plays an irreplaceable role in modern manufacturing. The inherent unstructuredness of the assembly environment and components poses a major challenge to the adaptability and accuracy. Existing approaches primarily target structured scenarios where the screw hole axis is perpendicular to the workpiece surface. These methods encounter significant challenges in accurately estimating the pose of unstructured screw holes, particularly in scenarios where the holes exhibit countersinks and the hole axes form arbitrary angles with the workpiece surface. To solve the problem, this paper proposes a two-stage progressive screw hole localization method, which estimates the hole pose through coarse localization followed by fine correction. In the coarse localization stage, target detection and edge segmentation are used to extract pixel points along the countersink edge, from which a coarse pose is estimated using spatial projection fitting. In the fine correction stage, a visual servoing network based on pose error classification is employed to iteratively refine the estimation and obtain an accurate screw hole pose. The proposed two-stage progressive positioning method effectively overcomes the limitations of pose estimation in unstructured environments. Its performance has been validated in real-world scenarios.