PanSR: An Object-Centric Mask Transformer for Maritime Panoptic Segmentation

Environmental panoptic perception and obstacle detection are key components in enabling autonomy for autonomous surface vehicles (ASVs). Recent studies have demonstrated that general panoptic methods, currently dominated by mask transformer approaches, are unsuitable for maritime environments, as they perform poorly in crowded scenes and in scenes with a wide range of object scales, highlighting the need for domain-specific panoptic methods. In this work, we identify several fundamental shortcomings of these recent methods and rethink architecture design choices from the perspective of perception for autonomous navigation. We introduce PanSR, a new panoptic segmentation method with the following main contributions: 1) an object-centric proposal extraction module for better handling of object size variety, 2) an improved ground-truth matching scheme consistent with proposal extraction, and 3) object-centric mask prediction for better instance separation of similar objects. With these enhancements, PanSR effectively mitigates instance merging, enhances small-object detection and increases performance in crowded scenes, delivering a notable +3.4% PQ improvement over state-of-the-art on the challenging LaRS benchmark and +8.4% PQ on WaterScenes, while retaining competitive performance on the autonomous driving perception benchmark Cityscapes. The code and models are publicly available on GitHub.