Dynamic Attention and Skeleton-Guided Contrastive Learning for Fine-Grained Roof Segmentation

Building roof types reveal important geometric cues for urban analysis. However, small inter-class differences make fine-grained roof segmentation challenging. Roofs often share similar textures and materials in overhead views, which makes it difficult for conventional instance segmentation models to capture the subtle geometric cues. To tackle these challenges, we propose the Fine-grained Roof Segmentation Network (FRS-Net), which incorporates specialized modules to enhance discriminative structural features tailored for roof geometries. First, we introduce a pair-specific attention module that dynamically activates specific attention patterns based on the most confused roof category pairs. By emphasizing subtle class-specific cues, this module effectively mitigates feature confusion among visually similar roofs. Second, we design a skeleton-guided contrastive learning module that uses synthetic roof skeletons as structural priors to guide feature embeddings. By treating same-type skeletons as positive pairs and different-type skeletons as negative pairs, the module drives the network to emphasize fine-grained geometric cues essential for distinguishing visually similar roof classes. Comprehensive experiments conducted on the UBCv1 and UBCv2 datasets demonstrate that FRS-Net achieves state-of-the-art results, boosting the mAP and AP50 metric to 17.0% and 28.6% on UBCv2 dataset, respectively. The source code will be released at https://github.com/wangyx-chn/FRS-Net.