CNN Compression via Channel-Wise Variance-Based Filter Pruning

In recent years, the focus on lightweight convolutional neural networks has led to extensive research in pruning techniques to reduce model size and inference cost while maintaining accuracy. Traditional structured pruning methods that rely on magnitude-based metrics, such as L1/L2 norms or geometric median, often produce similar filter rankings. This raises concerns about the uniqueness and informativeness of the resulting importance scores. In this study, we introduce a variance-based pruning method that diverges from conventional magnitude-based approaches by evaluating filter importance through the variance distribution of weights within convolutional layers. Our theoretical analysis shows that weighted variance captures complementary information overlooked by norm-based methods, distinguishing our criterion from existing approaches. To validate the effectiveness of our approach, we conduct comprehensive experiments with the ImageNet dataset. We compare our method against multiple baseline pruning techniques – including L1-norm pruning, L2-norm, geometric median, HRank, RASP, AutoSlim, and Taylor-approximation-based pruning. Notably, our approach achieves up to 76.2% Top-1 accuracy with only 2G MACs, which is less than half the computational cost of the ResNet-50 baseline.