VFM-CAKD: Category-Aware Knowledge Distillation From Visual Foundation Model for Aerial Imagery Semantic Segmentation

Aerial image semantic segmentation is a cornerstone technology for low-altitude economy applications, but lightweight models-indispensable for real-time inference on resource-constrained platforms like drones-suffer from limited feature representation capacity. Visual Foundation Models (VFMs), with robust and generalizable visual semantics learned from massive data, offer great potential for enhancing lightweight models via knowledge distillation (KD). However, direct application of VFM-based KD faces three unaddressed challenges: task-irrelevant knowledge interference in VFMs, inherent intra-class variations in aerial imagery, and inadequate long-range dependency modeling of lightweight networks. To tackle these issues, we propose VFM-CAKD, a category-aware knowledge distillation method tailored for lightweight aerial image semantic segmentation. First, we introduce learnable category-aware vectors to filter task-irrelevant features from VFMs, generating Category-Aware Guidance (CAG) to prioritize task-critical semantics. Second, a Category-Aware-Guided Dual-Scale Feature Fusion (CAG-DSFusion) module is designed to activate category-aware representations, extracting stable intra-class features and robust prototypes to mitigate intra-class variations. Third, a Category-Aware-Guided Global Attention (CAG-Global) module aligns teacher-student global attention maps, transferring VFMs' long-range dependency modeling capability. Experiments on drone-view datasets (Aeroscapes, UAVid), aerial remote sensing datasets (Potsdam, Vaihingen), and a disaster-scenario dataset (RescueNet) demonstrate that VFM-CAKD effectively transfers task-relevant VFM knowledge, significantly improving the performance of lightweight segmentation models. The code is avaliable athttps://github.com/aresdrw/VFM-CAKD.