RGB-to-NIR Facial Image Translation via Lightweight Generative Adversarial Network

In surveillance systems, near-infrared (NIR) images of faces are more suitable for nighttime use than RGB images. However, the NIR images of Computer-Generated (CG) characters cannot be obtained directly through photography. One possible solution is to estimate NIR images from corresponding RGB images using image-to-image (I2I) translation. The development of algorithms also requires the support of sufficient datasets. While large-scale datasets from the visible spectrum are available, the lack of NIR counterparts has posed a significant obstacle to related research. To address the issues mentioned above, we propose a generative adversarial network (GAN)-based I2I translation model that generates NIR human faces from RGB inputs. The U-shaped generator can recover semantic and texture information with customized feature extraction and skip connection modules. The generator’s parameters are 10.54M, nearly one-fifth of the strong baseline FRAGAN. It outperforms the strong baseline under five representative image quality evaluation criteria: SSIM, PSNR, NRMSE, LPIPS, and FID by 0.009, 0.791, 0.008, 0.040, and 7.967 on the small-scale dataset. In terms of visualization, the facial details show apparent NIR features, proving that the generated images are of high visual quality. Finally, the trained generator can produce possible NIR photographs of the CG characters under the same image acquisition conditions as the training dataset.