Multi-Level Online Group Pruning for Deep Semantic Segmentation: Balancing Efficiency and Class Fairness

This study targets Encoder-Decoder architectures acceleration for semantic segmentation. Although deep neural networks have shown superior performance in image segmentation tasks, their substantial model size and high computational complexity limit their application in real-time processing and resource-constrained devices. To tackle this issue, pruning techniques have been developed to compact Encoder-Decoder networks. However, most current pruning methods are designed for classification tasks, rendering them challenging to achieve satisfactory results on fine-grained demands of segmentation tasks. Additionally, these methods only consider the filters’ global average contribution to each class, which can compromise model sensitivity to minority classes. To solve this issue, we propose a multi-level online group pruning method. Our method identifies and preserves the most discriminative filters for critical minority classes, ensuring that the pruned network can still accurately segment these underrepresented categories. Consequently, it fosters a more balanced and fair segmentation performance across all classes.We evaluated our method on two popular image segmentation datasets, i.e., Camvid and Cityscapes. After pruning 93% of parameters in trendy Encoder-Decoder networks, our method achieves noteworthy IoU gains for the minority classes, e.g., 6.5% for Pole on Camvid and 4.7% for Fence on Cityscapes, over state-of-the-art online pruning methods, while still maintaining comparable or superior performance for majority classes.