Introducing WSOD-SAM Proposals and Heuristic Pseudo-Fully Supervised Training Strategy for Weakly Supervised Object Detection in Remote Sensing Images

Weakly supervised object detection (WSOD) in remote sensing image (RSI) relies solely on image-level labels, thereby significantly reducing the cost of instance-level annotations. Recently, pseudo-fully supervised object detection (PFSOD) models trained with image-level labels have achieved superior performance compared to traditional WSOD models. However, two major challenges remain. Firstly, the pseudo ground truth (PGT) used in existing PFSOD models often suffers from low localization accuracy, resulting in suboptimal detection results. Secondly, existing PFSOD models tend to miss detections when multiple objects of the same category exist within an RSI. To address these issues, we propose a novel PFSOD framework incorporating two key innovations. To handle the first challenge, a weakly supervised object detection guided segment anything model (WSOD-SAM) is introduced to generate object proposals instead of traditional selective search algorithms based on low-level features. The WSOD-SAM proposals can combine the detection capability of the WSOD model with the segmentation capability of SAM, therefore, the baseline WSOD model equipped with them can produce more accurate PGTs. To handle the second challenge, we design a heuristic pseudo-fully supervised training (HPFST) strategy, which dynamically mines potential objects with high class confidence scores as pseudo soft labels for the classification branch of the YOLO model, in addition to using the PGT. Ablation studies confirm the individual and combined effectiveness of WSOD-SAM proposals and HPFST strategy. Extensive experiments demonstrate that our method achieves 78.12% and 31.44% mAP on the NWPU VHR-10.v2 and DIOR datasets, respectively, outperforming 16 existing WSOD methods. Our code is available at https://github.com/ccu6/hps-yolo.