Sem-GAP: Enhancing Cross-Category Pose Estimation with Vision Foundation Models and Conditional Diffusion Models

View at publisher Summarise with VIRAT
Authors: Tongchuan Xia, Chao Wan, Yushi Chen, Quanzhen Huang
Cited by: 0
References: 10

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Robotic manipulation in embodied intelligence relies on precise pose perception of cross-category Generalizable and Actionable Parts (GAParts). However, when handling objects from unseen categories, existing deterministic regression methods face multi-modal pose ambiguity induced by rotational symmetry, often yielding physically infeasible mean predictions. To address this, this paper proposes the Sem-GAP framework. This method first constructs a multi-modal feature fusion mechanism that performs point-wise concatenation of high-dimensional semantic representations from Vision Foundation Models and geometric features from point clouds via dense correspondence, thereby enhancing generalization capabilities for unseen categories. Building upon this, we introduce a conditional diffusion model to formulate pose estimation as a probabilistic distribution sampling process, utilizing global geometric-semantic conditions to guide the generation of physically constrained poses. Experiments on the GAPartNet benchmark and physical simulations demonstrate that our method significantly improves pose estimation accuracy for unseen categories and symmetric parts, increasing the average success rate of downstream robotic manipulation tasks by 18.4%.

Fields of Study & Subjects

Robot Manipulation and Learning 3D Shape Modeling and Analysis Advanced Neural Network Applications

Publication details

Journal / source
International Conference on Communication Networks and Machine Learning (CNML)
Pages
889–892
Publication date
30 January 2026
Language
EN