HENet: A Heterogeneous Encoding Network for General and Robust Adversarial Example Generation

Generator-based adversarial attack methods aim to fool deep neural networks (DNNs) by training a generator for crafting adversarial examples (AEs). However, as DNNs evolve from Convolutional Neural Networks (CNNs) to Transformers, the existing generator-based methods can hardly achieve satisfactory attack performance against different target model architectures in semi-whitebox attack scenarios. In addition, the generated AEs are susceptible to various distortions (especially for JPEG compression with low quality factors), which deteriorate the attack ability and increase the unreliability. To address these issues, we propose a dual-branch guided generative model called Heterogeneous Encoding Network (HENet) to form a robust generator-based adversarial attack framework. Specifically, our HENet introduces an Adaptive Feature Fusion Module (AFFM) to solve the dimensions and representativeness contradictions between CNNs and Transformers, which steers the perturbation generation based on a richer latent space and achieves better general attack ability. To further improve the robustness against JPEG compression, we design and integrate a Dynamic Differentiable JPEG Simulator (DDJS), which introduces an adaptive quantization mask to determine the flow of the gradient backpropagation in each frequency position. Extensive experiments prove the proposed method achieves a better attack success rate, lower perturbation magnitude, and higher robustness for various target network architectures under compressed, distorted, and lossless scenarios. Our codes will be made publicly available.