Modeling Image Segmentation Label Noise Through Concave Region Removal

The effects of label noise geometry on image segmentation model training are under-studied, in part because prevailing segmentation metrics such as the Intersection over Union are based on pixel counts and cannot describe spatial properties of label noise. It is well established that the degree of label noise randomness affects training, however, the effects of non-random geometric label noise have not been evaluated. To this end, we describe a new class of non-random geometric label noise characterized by removal of concave regions from connected components in segmentation masks. This class of label noise geometry is motivated by the observation that both human and automated annotations have vulnerabilities that disproportionately affect concave regions. We validate this class of label noise by demonstrating its prevalence in a dataset containing noisy crowdsourced labels for the PASCAL, BDD100K, and COCO benchmarks. We then show that concave region label noise affects model training differently than uniformly applied geometric transformations across a range of model architectures and image domains. These results motivate the study of new segmentation metrics better tuned to detect label noise geometries most detrimental to model training.