Hyperspectral Weak Object Detection Using Homogeneous-Unique Feature Fusion

Hyperspectral object detection (HOD), as an emerging research field, has attracted growing attention due to its advantage for detecting objects in challenging scenes with weak objects. However, existing methods mainly depend on off-the-shelf RGB detectors for feature extraction, failing to fully exploit the abundant spectral context information in hyperspectral images (HSIs), which results in suboptimal detection performance, especially in the case of weak objects such as tiny or camouflaged objects. In this paper, a method named HYPER-DET is proposed. Firstly, to address the issue of insufficient utilization of spectral context information in feature extraction, a novel cross-modality feature extraction network (CFEN) is introduced to extract visual and spectral features simultaneously, thus enhancing the feature representation of weak objects. In the RGB branch of the proposed backbone, an end-to-end self-sensing regrouping module (SSRM) is presented to capture cross-band correlations and cluster optimal three-channel spectral band groups. In the HSI branch, a spatial-spectral feature extraction module is proposed to fully extract the complementary spatial and spectral information from HSIs. Secondly, to facilitate the efficient fusion of the cross-modal features between RGB and HSI modalities, a homogeneous-unique feature fusion (HUFF) module is designed, which adaptively extract and fuse both homogeneous and unique features across these two modalities. Finally, we introduce a new hyperspectral aerial tiny object dataset, HWOD-tiny. It is tailored for challenging scenarios that involve tiny Unmanned Aerial Vehicle (UAV) objects, providing a comprehensive benchmark for evaluating HOD methods in competitive settings. HWOD-tiny is the first dataset designed for hyperspectral aerial tiny object detection. Extensive experiments on multispectral UAV imagery dataset DroneVehicle, general HOD dataset HOD3K and the proposed datasets demonstrate the effectiveness of the proposed HYPER-DET, which outperforms existing state-of-the-art object detection methods. The source datasets in the article will be available athttps://github.com/m1ftb1/hyperdet.