PPOM-Attack: A Substitute Model-Free Perturbation Prediction and Optimization Method for Black-Box Adversarial Attack Against Face Recognition

Face recognition (FR) brings convenience to peoples lives while also posing security risks. Some malicious users employ FR attacks to impersonate the identity of a target. To reveal the security risks, recent work has attacked black-box FR models by utilizing substitute models to generate adversarial face images that are misclassified as the target individual due to the attack transferability of substitute models. However, the substitute models cannot accurately approximate the target model that leads to a decrease in FR attack success rate and adversarial face image quality. To address the issue, we propose the PPOM-Attack, a substitute model-free Perturbation Prediction and Optimization Method for black-box adversarial Attack against face recognition. PPOM-Attack directly obtains feedback from the target model instead of using substitute models, it avoids any discrepancy with the attack objective. To achieve this goal, we design a proximal policy optimization (PPO)-based agent to predict the perturbation regions in the face image and self-adaptively disturb the regions. To maintain high-quality adversarial face images, we further propose a minimum brightness offsets method specifically designed to generate perturbations that minimize the feature embedding difference between the adversarial and targeted face images. The experimental results show that our approach outperforms state-of-the-art FR attack methods by an average of 21.7% in terms of attack success rate, while achieving better image quality on seven FR models.