A hierarchical deep reinforcement learning method for dragging and adjusting objects with dual-arm robot

In spite of the ease with which humans can pick up and use objects, object manipulation remains a popular and challenging research topic even after many decades. The object’s shape, weight, or position can make it impossible for the robot to manipulate it with a single arm. In such situations, dual-arm manipulators are needed. Furthermore, manipulation may require dragging or pushing objects. Dragging an object while considering the desired orientation of the object, along with uncertainties like friction on the surface and possible slipping of the object in the robot’s hands, is very challenging. Therefore, in this paper, we introduce a novel hierarchical deep deterministic policy gradient (HDDPG) that exploits the continuity of the state and action spaces based on the actor-critic, model-free algorithm as a strategy controller to solve the dual-arm object dragging problem. To evaluate the proposed algorithm, we conduct extensive experiments both in simulation and on a real adult-sized humanoid robot. We use 13 different objects, including keyboards, laptops, boxes, etc. These experiments demonstrate the effectiveness and high performance of the proposed algorithm, with an average success rate of 97.3% in simulations and 93.84% and 91.69% in real environments, with 1560 attempts on two different surfaces.