PhysiXDeform: Real-Time Vision-Guided Soft-Tissue Deformation Prediction With Physical Priors

Accurate prediction of soft tissue deformation from endoscopic video is critical for robot-assisted, image-guided interventions. However, it remains challenging due to occlusions, complex dynamics, and the absence of direct physical measurements. Existing vision-based approaches often impose rigid or simplified motion priors, limiting their generalisability and physical interpretability. In this paper, we introduce a vision-guided, physics-informed framework that predicts multi-step 3D deformation by integrating stereo disparity, optical flow, and tracked keypoints into a spatio-temporal graph. Motion features, including velocity, acceleration, and directional strain, are encoded as node and edge attributes and processed via a graph network. The model is trained end-to-end with self-supervised objectives promoting geometric alignment, temporal coherence, and motion consistency. Across synthetic and real surgical datasets, our method achieves up to28.8%reduction in Chamfer Distance (CD) at short prediction horizons, and an average43.9%reduction over extended-horizon forecasts. In cross-scene adaptation, our model achieves average CD reductions of12.9%with light tuning and12.2%with heavy tuning, where the latter shows slight overfitting but still outperforms mesh-based baselines and supports real-time operation.