PLVAM-DETR: Patch-Level Visibility Aware Multi-Spectral Detection Transformer With Frequency Specific Fusion

View at publisher Summarise with VIRAT
Authors: Xiuwei Zhang, Haorui Zeng, X.S. Zhang, Wencong Wu, Hang Yin, Shun Dai, Yanning Zhang
Cited by: 0
References: 51

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Multi-spectral object detection demonstrates enhanced robustness compared to single-spectral object detection by leveraging complementary information from both visible and infrared spectra. Current presented methods often fuse multi-spectral information using attention mechanisms, such as spatial/channel attention for CNN-based approaches and cross-attention/self-attention for Transformer-based approaches. While, object visibility across different regions in different spectral images is varying. Therefore, explicit patch-level visibility-awareness is required to perform finer spatial and spectral exploration. Moreover, the distinct frequency characteristics of infrared and visible images are rarely highlighted, hindering effective utilization of their complementary benefits. To overcome these limitations, we propose a patch-level visibility aware multi-spectral detection Transformer with frequency specific fusion, named PLVAM-DETR. A patch-level visibility aware module is designed to dynamically determine the significance across different patches in different spectral images. Then, a frequency specific feature fusion module is presented to highlight high-frequency information in the infrared features and low-frequency information in the visible features, which provides more comprehensive feature for detection. Extensive experiments on publicly available datasets demonstrate competitive results compared to state-of-the-art methods.

Fields of Study & Subjects

Advanced Neural Network Applications Advanced Image Fusion Techniques Advanced Image and Video Retrieval Techniques

Publication details

Journal / source
IEEE Transactions on Multimedia
Volume
28
Pages
9009–9019
Publication date
1 January 2026
Language
EN