Visible-infrared imaging fusion method based on low light enhancement for factory night monitoring

In the planning of intelligent factories, high-quality monitoring in nighttime scenarios is currently a key challenge. Existing cameras typically use visible (VIS) and long wavelength infrared (LWIR) dual bands image fusion methods for night vision enhancement. VIS images can display rich texture details, while LWIR images have high contrast characteristics. However, most prior works on VIS-LWIR images fusion are designed for normal nighttime conditions. To address this issue, this article proposes a VIS-LWIR fusion method by combining low light VIS image enhancement tasks. We firstly designed a low light enhancement network to enhance VIS images in ultra-low light environments while ensuring color consistency of the images. Then a heterogeneous image fusion network was designed to integrate complementary information from the source images, and a dual gradient residual module was designed to improve the fusion network’s capability to describe fine-grained spatial details. Finally, we fully considered the intrinsic relationship between low-light image enhancement and image fusion, and designed the loss function as a joint loss consisting of illumination enhancement loss and fusion loss. The experimental results on LLVIP dataset show the outperformance of the present method in the metrics of mutual information (MI), visual information fidelity (VIF), spatial frequency (SF), average gradient (AG), and quality assessment based on blur and nose factors (Qabf), and second place in the metrics of entropy (EN). These results indicate that the proposed method effectively enhances the imaging quality under ultra-low light conditions.