Sem-GAP: Enhancing Cross-Category Pose Estimation with Vision Foundation Models and Conditional Diffusion Models
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Robotic manipulation in embodied intelligence relies on precise pose perception of cross-category Generalizable and Actionable Parts (GAParts). However, when handling objects from unseen categories, existing deterministic regression methods face multi-modal pose ambiguity induced by rotational symmetry, often yielding physically infeasible mean predictions. To address this, this paper proposes the Sem-GAP framework. This method first constructs a multi-modal feature fusion mechanism that performs point-wise concatenation of high-dimensional semantic representations from Vision Foundation Models and geometric features from point clouds via dense correspondence, thereby enhancing generalization capabilities for unseen categories. Building upon this, we introduce a conditional diffusion model to formulate pose estimation as a probabilistic distribution sampling process, utilizing global geometric-semantic conditions to guide the generation of physically constrained poses. Experiments on the GAPartNet benchmark and physical simulations demonstrate that our method significantly improves pose estimation accuracy for unseen categories and symmetric parts, increasing the average success rate of downstream robotic manipulation tasks by 18.4%.
Fields of Study & Subjects
Robot Manipulation and Learning
3D Shape Modeling and Analysis
Advanced Neural Network Applications
Publication details
- Journal / source
- International Conference on Communication Networks and Machine Learning (CNML)
- Pages
- 889–892
- Publication date
- 30 January 2026
- Language
- EN