Real-Time Simulation of Deformable Tactile Sensors and Objects in Robotic Grasping using Graph Neural Networks with Inductive Biases

Physical simulation of deformable bodies is crucial for robotic manipulation, particularly for applications involving deformable objects and deformable tactile sensors. While Finite Element Method (FEM) simulations provide high accuracy for modeling deformable objects and tactile sensors, they are prohibitively expensive for real-time applications, with simulation times often exceeding practical limits for robotic control and learning. This paper presents a novel Graph Neural Network (GNN) framework that accelerates the simulation of tactile sensors by factors of $10^{2}-10^{4}$ compared to FEM, while maintaining high physical accuracy. Our approach addresses limitations in existing GNN-based physics learning through inductive biases. The key contributions include: (1) extending FEM simulation to deformable tactile sensors in grasping scenarios, (2) incorporating novel inductive biases through tetrahedral features and global graph features to improve information propagation and training stability, and (3) demonstrating the first successful application of GNN simulation for tactile sensors with generalization to unseen objects. Additionally, the inductive biases reduce prediction errors by up to 45% compared to baseline approaches. This work enables real-time simulation of soft tactile sensors and deformable objects for robotic applications, including deformation and stress prediction. All simulation and training code can be found at https://tacgraspnets.github.io/.