PLVAM-DETR: Patch-Level Visibility Aware Multi-Spectral Detection Transformer With Frequency Specific Fusion
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Multi-spectral object detection demonstrates enhanced robustness compared to single-spectral object detection by leveraging complementary information from both visible and infrared spectra. Current presented methods often fuse multi-spectral information using attention mechanisms, such as spatial/channel attention for CNN-based approaches and cross-attention/self-attention for Transformer-based approaches. While, object visibility across different regions in different spectral images is varying. Therefore, explicit patch-level visibility-awareness is required to perform finer spatial and spectral exploration. Moreover, the distinct frequency characteristics of infrared and visible images are rarely highlighted, hindering effective utilization of their complementary benefits. To overcome these limitations, we propose a patch-level visibility aware multi-spectral detection Transformer with frequency specific fusion, named PLVAM-DETR. A patch-level visibility aware module is designed to dynamically determine the significance across different patches in different spectral images. Then, a frequency specific feature fusion module is presented to highlight high-frequency information in the infrared features and low-frequency information in the visible features, which provides more comprehensive feature for detection. Extensive experiments on publicly available datasets demonstrate competitive results compared to state-of-the-art methods.
Fields of Study & Subjects
Advanced Neural Network Applications
Advanced Image Fusion Techniques
Advanced Image and Video Retrieval Techniques
Publication details
- Journal / source
- IEEE Transactions on Multimedia
- Volume
- 28
- Pages
- 9009–9019
- Publication date
- 1 January 2026
- Language
- EN