Towards lightweight noise-robust models via noise-aware pruning

Model pruning aims to reduce the computational cost and space consumption of deep neural networks while maintaining their performance. It is essential for deploying deep models on resource-constrained edge devices. With increasing attention to learning with noisy labels (LNL), achieving lightweight deployment of noise-robust models has become an urgent demand. However, existing pruning approaches are mostly designed for ideal scenarios where all training samples are accurately labeled. Therefore, their effectiveness becomes limited when applied to noise-robust deep networks trained with noisy labels. To this end, we propose a simple yet effective method, termed NAP (Noise-Aware Pruning), to prune noise-robust models. Specifically, we begin by identifying clean and noisy sample subsets from the noisy training data, utilizing both feature-based and prediction-based perspectives. Then, we introduce a composite pruning criterion that evaluates the parameter importance based on the degree of parameter fitting to these two data types. Lastly, based on this criterion, noise-robust models are selectively pruned to enhance their efficiency. Comprehensive experimental results and ablation studies on various models and noisy datasets are provided to demonstrate the effectiveness of our proposed method. Our source code has been made available at https://github.com/t1748917720/Noise-Aware-Pruning.