A Contact-Model-Guided Visuotactile Sensor Enabling Simultaneous Measurement of Hardness and Tactile Information

Haptics plays a crucial role in enabling robots to achieve safe and coordinated interactions with their surroundings. Among the diverse range of tactile sensors, visuotactile sensors have emerged as leading candidates for sophisticated fingertip sensing applications due to their high resolution and capability to perceive multimodal tactile information. However, current visuotactile sensors face limitations in durability and the ability to rapidly measure multiple tactile information simultaneously during brief contact similar to human touch. We propose a visuotactile sensor guided by contact models, through our investigation of Hertzian contact and planar contact. Specifically, the contact model has revealed the relationship between contact force, depth, area, and the hardness of the contacting objects under contact objects of varying shapes. By embedding these physical priors into a fitting framework, our method avoids data-intensive learning, enabling real-time, simultaneous estimation of 3-axis force, deformation depth, and Shore A hardness. For three tests on the same object, the average error in normal force is around 1.3 N and the root mean square error in hardness is 7.656 on the Shore A scale. At the same time, we have enhanced the material and processing methods of the visuotactile sensor’s functional layer to improve its stability and durability, so that it can be adapted to objects of various hardness.