Revisiting the Adversarial Transferability: Toward a Perspective of Semantic Preservation

Transfer-based attacks pose a significant threat to deep neural networks, as they can attack without accessing the target model. Recently, input transformation-based attacks have emerged as a research hot-spot in transfer-based attacks. This is attributed to their efficiency in enhancing the transferability of adversarial examples. In this work, we revisit and reveal that recent input transformation-based attacks mainly introduce a large variation, to the extent that they inadequately preserve the semantic information. Such semantic inconsistency results in a misalignment between the gradient information for generating perturbations and the true decision boundaries of target models, intensifying overfitting to the surrogate model due to inaccurate gradients, which limits the transferability. To tackle this problem, we propose a novel input transformation-based attack method called Selfmix, which employs a localized mixing paradigm to mix various regions of the input image itself. Our Selfmix enhances the diversity of transformed images while effectively preserving semantic consistency, thereby improving the transferability. Moreover, Selfmix demonstrates effective compatibility with other input transformation-based attacks due to enhanced semantic preservation. Comprehensive experiments on the ImageNet dataset verify the competitiveness of Selfmix in enhancing transferability on both normally trained models and defense models, and the generality to various deep network architectures, especially when attacking transformer-based models. The code is publicly available athttps://github.com/bookman233/SelfMix.