CoordAR: One-Reference 6D Pose Estimation of Novel Objects via Autoregressive Coordinate Map Generation

Object 6D pose estimation is a challenging task that is crucial for robotics and augmented reality applications, particularly when dealing with novel objects. A promising direction is single-reference-based estimation, which requires only a single annotated view instead of a full 3D model. However, existing methods rely on dense correspondence regression, which suffers from limited global consistency due to the local nature of convolutional architectures, and faces challenges in symmetric or occluded scenarios due to deterministic predictions. We present CoordAR, a novel autoregressive framework for single-reference 6D pose estimation of unseen objects. CoordAR formulates 3D-3D correspondences between the reference and query views as a discretized coordinate map, which is decoded autoregressively in a probabilistic manner. To enable accurate correspondence regression, CoordAR introduces: 1) a novel coordinate map tokenization enabling probabilistic prediction over discretized 3D space; 2) a decoupled encoding strategy that separately encodes RGB appearance and coordinate cues; and 3) an autoregressive transformer decoder conditioned on both pixel-aligned query features and the partially generated coordinate sequence. Thanks to the novel designs, CoordAR significantly outperforms existing methods on multiple benchmarks and demonstrates strong robustness to symmetry, occlusion, and other challenges in real-world tests, while requiring only a single reference view.