Skin Classification for Face Recognition Based on Deep Learning with U-Net and ResNet

Face recognition under uncontrolled lighting remains challenging due to variations in brightness, background noise, and low-quality features. This paper presents a unified deep learning model that integrates illumination normalization, skin-aware spatial modulation, and quality-based margin learning within a single inference process. Unlike earlier methods that treat relighting or segmentation as preprocessing, this approach directly integrates mask-guided feature modulation into embedding learning. The system comprises RetinaFace detection, photometric augmentation during training, lightweight neural relighting at inference, U-Net-based skin segmentation, and identity embeddings trained with ArcFace, AdaFace, or MagFace losses, with angular margins adapted to feature quality. Experiments on Labeled Faces in the Wild (LFW), Celebrities in Frontal-Profile (CFP-FP), Age Database 30 (AgeDB-30), and a custom illumination dataset demonstrate steady enhancements in difficult lighting conditions. The model reaches a competitive 99.8% accuracy on LFW and shows notable improvements on pose-hard CFP-FP and the custom dataset, such as a +2.6% increase in TPR at 1 × 10−4 FPR. The key innovations include: (i) mask-guided embedding modulation that embeds segmentation into feature learning, (ii) a dual strategy combining training-time photometric data augmentation with inference-time neural relighting, and (iii) joint spatial–quality margin learning via AdaFace/MagFace. Finally, results confirm consistent gains under challenging illumination and pose variations.