Synchronous visual servoing and obstacle avoidance of manipulator based on dynamic matrix control

Abstract Obstacle avoidance planning is essential for manipulators when performing visual servoing in unstructured environments, particularly with a large camera displacement. Visual servoing focuses on the motion of the end‐effector (or the eye‐in‐hand camera), while obstacle avoidance planning focuses on the motion of the manipulator's body. In previous research, these two tasks have typically been regarded as independent. In this article, a new controller is proposed to perform visual servoing and obstacle avoidance simultaneously. The proposed controller is based on dynamic matrix control (DMC), with the image‐based visual servoing (IBVS) principle serving as its predictive model. A modified artificial potential field (MAPF) is introduced to dynamically update the control weight matrix of the DMC for obstacle avoidance. Based on the proposed controller, an IBVS system is implemented for precise manipulation in unstructured environments. The effectiveness and advantages of the proposed controller are demonstrated through simulation and real‐world experiments.