Efficient Face Recognition Using Embedding-Based Distillation

Knowledge distillation (KD) facilitates the compression of large, high-performing neural networks into efficientstudent models, enabling deployment on resource-limited devices like mobile phones and IoT systems. This paperintroduces a KD methodology, which involves training a student to capture a teacher’s soft labels, intermediate featurerepresentations, and ground truth labels, ensuring both compactness and accuracy. Applied to face recognition, our hybridKD framework trains a MobileFaceNet student under an InceptionResNetV1 teacher, achieving 90.40% accuracy and a96.25% AUC, outperforming lightweight models while remaining suitable for edge devices. These results highlight thepotential of KD to enable robust, scalable face recognition solutions for real-world, resource-constrained environments.