Cooperative Control Framework for Dual-Arm Robot Enhanced by Vision Language Model and Reinforcement Learning

This paper presents a cooperative control framework for dual-arm robots that integrates vision-language models (VLMs) with online reinforcement learning (RL) to enhance autonomy and adaptability in complex manipulation tasks. The proposed framework adopts a hierarchical architecture: at the top level, the VLM interprets natural language instructions and visual image to generate task plans; at the middle level, an online RL module refines manipulation policies and ensures adaptive decision-making under environmental uncertainty; and at the bottom level, compliant control based on trajectory planning and impedance regulation enables safe and robust execution. In the feedback, YOLOv5 is used to detect the object, GraspNet is used to obtain the optimal grasp pose, and CLIP (Contrastive Language-Image Pre-Training) is used to judge whether task is completed. Simulations and real-world experiments validate the effectiveness of the proposed method. The dual-arm robot successfully performed various cooperative tasks such as grasping, bottle-cap unscrewing, water pouring, and box carrying, achieving an increase in the task success rate from 43% to 100% with online adaptive learning and training. These results demonstrate that the proposed framework effectively bridges high-level reasoning with low-level control, providing a scalable solution for future applications in service robotics, industrial automation, and human-robot collaboration.