Novel Class Object Detection with Object-Centric Data Synthesis (Student Abstract)

Deep learning approaches to object detection have achieved reliable detection of specific object classes in images. However, extending a model’s detection capability to new object classes requires large amounts of annotated training data, which is costly and time-consuming to acquire, especially for long-tailed classes with insufficient representation in existing datasets. We compare four distinct methods of generating synthetic data to finetune object detection models on novel object categories, particularly when limited data is available in an object-centric format (multi-view images/3D models). Our approaches are based on simple image processing techniques, 3D rendering, and image generation models, each varying in complexity and realism. We assess how our methods, which use object-centric data to synthesize realistic, cluttered images with varying contextual coherence, enable models to achieve category-level generalization in real-world data. We demonstrate significant performance boosts within this data-constrained experimental setting.