Hierarchical Category Refinement for remote sensing segmentation: A new task, benchmark datasets, and the CG-HCAN framework

The proliferation of high-resolution remote sensing imagery offers unprecedented detail for Earth observation, yet its potential is constrained by the coarse granularity of existing semantic labels. This discrepancy limits fine-grained analysis and highlights a critical research gap: the absence of automated methods for refining coarse annotations into multi-level, semantically consistent labels. To address this, we introduce the novel task of Hierarchical Category Refinement (HCR), which focuses on the programmatic evolution of existing coarse labels into a multi-level, fine-grained semantic hierarchy, and propose the CLIP-Guided Hierarchical Category-Aware Network (CG-HCAN). This framework features a dual-stream architecture with a Bidirectional Cross-Attention Fusion (BiCAF) module to effectively integrate image features and coarse boundary priors. At its core, CG-HCAN integrates a CLIP-guided semantic graph neural network that leverages semantic knowledge distilled from a pre-trained vision–language model to drive fine-grained category discrimination and relational reasoning. To validate our approach, we constructed two large-scale, three-level benchmark datasets, EVLAB-HCR and GID-HCR. Extensive experiments demonstrate CG-HCAN outperforms strong baselines adapted from state-of-the-art segmentation methods. Specifically, on the GID-HCR benchmark, our fine-grained Level 3 (L3) mIoU surpasses the next-best method by 2.81 percentage points, while maintaining near-perfect hierarchical consistency (99.91%). This work not only establishes a new benchmark and a robust methodological framework for the HCR task but also provides a practical solution for automatically upgrading the semantic granularity of existing remote sensing datasets, enabling more detailed and accurate land cover analysis. The source code and the two benchmark datasets will be publicly available at https://github.com/WHUXyb/CG-HCAN.