Red green blue-depth multimodal learning for vision-guided grasping in Delta robots

Abstract In unstructured environments, Delta robots face challenges in achieving high vision-guided grasping precision due to dynamic lighting conditions and workpiece diversity. This paper designs an integrated solution that combines RGB-D multimodal learning with an enhanced Mask R-CNN framework. Initially, a dual-stream ResNet50-FPN backbone network is designed to achieve cross-modal adaptive alignment via hierarchical feature fusion. Subsequently, a depth-guided attention module is incorporated to bolster robustness against material ambiguity and reflective interference. Moreover, a dynamic depth estimation algorithm is employed to significantly improve target localization accuracy and stability. Finally, real-time trajectory tracking is realized by integrating PD control with Jacobian mapping. Experimental results validate the efficacy of the proposed method, offering an efficient and reliable approach for industrial robotic applications.