Relative Geometrical Constraint on Finger Motion for Dexterous Teleoperation of Multifingered Hand

This paper presents a method that enables intuitive and stable in-hand manipulation in teleoperation of a multifingered robotic hand. Our method comprehensively handles the fundamental components of in-hand manipulation, such as object pose control, finger sliding on an object, and finger gaiting. This is achieved by constraining the operator’s finger commands relative to the intended object motion, preventing their penetration into the object. Moreover, this method does not rely on any visual-based sensors and instead utilizes the six-axis force-torque sensors at the fingertips. The proposed method was evaluated with hardware through in-hand manipulation of a screwdriver, and the intuitive manipulation with high tracking performance of the operator’s finger motion was demonstrated. Additionally, we present imitation learning results using the data collected by the proposed method. Policy learned with constrained finger motion as expert action maintained the performance even without the constraint during policy execution. This shows that the dataset is scalable as it is usable even without implementing the proposed constraint.