From Local Matches to Global Masks: Template-Guided Instance Detection and Segmentation in Open-World Scenes

Candidate Points Local-to-Global Instance Detection (Ours) Proposal-Based Instance Detection Mask Reconstruction Detection Result (a).Detection Result (a).Detection Result (b).Detection Result (b).Query Images Templates … … Fig. 1: Conceptual comparison between object proposal-based instance detection methods and our local-to-global instance detection framework.Top: Proposal-based approaches [28]first generate object proposals in the query image and then perform instance matching to obtain detection results.Bottom: Our method starts from dense local correspondences to identify candidate points and reconstructs complete instance masks through mask reconstruction, producing final detection results without explicit proposal generation.Abstract-Detecting and segmenting novel object instances in open-world environments is a fundamental problem in robotic perception.Given only a small set of template images, a robot must locate and segment a specific object instance in a cluttered, previously unseen scene.Existing proposal-based approaches are highly sensitive to proposal quality and often fail under occlusion and background clutter.We propose L2G-Det, a localto-global instance detection framework that bypasses explicit object proposals by leveraging dense patch-level matching between templates and the query image.Locally matched patches generate candidate points, which are refined through a candidate selection module to suppress false positives.The filtered points are then used to prompt an augmented Segment Anything Model (SAM) with instance-specific object tokens, enabling reliable reconstruction of complete instance masks.Experiments demonstrate improved performance over proposal-based methods in challenging open-world settings.1