Framework for Robot Door Opening Based on Visual, Force, and Tactile Integration

A large number of robotic systems rely on visual perception to interpret their environment. However, depending solely on vision can lead to failures in manipulation tasks, particularly with constrained objects such as cabinet doors. While previous research investigated failure recovery in the context of handled doors, handleless furniture is becoming common in modern households and introduces new challenges that stem from narrow gaps in which the robot inserts the gripper for manipulation tasks. To address this, we propose a framework for autonomous opening of handleless cabinet doors that integrates visual, force, and tactile sensing to enhance robustness and enable failure recovery. The framework features a novel correction method that utilizes information from unsuccessful robot actions to correct camera parameters. These corrected parameters are used for updating the environment model, enabling the robot to replan a more reliable door-opening trajectory. We evaluate the correction method in simulation through an ablation study of key parameters and validate its performance on a real-world robot using a UR5 arm equipped with a 3D camera, a force-torque sensor, and a tactile sensor.