An FPGA-Efficient CNN Accelerator by Integrating Value-Based Sparsity and Bit-Level Sparsity for Weight Compression

Quantization and sparsity are key techniques for CNN compression and acceleration. Independent exploration of value-based sparsity (VBS) or bit-level sparsity (BLS) in 8b weights yield limited gains. We analyze weight distributions of VGG16 and ResNet50 after 4b quantization and find over 80% zero-values with strong near-structured sparsity. On VGG16, 98.11% of 16-weight groups contain at least 8 zero-values. Based on this analysis, we propose a two-stage sparse compression (TSSC) method that integrates VBS and BLS to efficiently compress weights. In stage 1, we apply post-quantization pruning to enforce structured sparsity with at least 8 zero-values per 16 weights and utilize value-based structured sparsity (VBSS) to compress these 8 zero-values per group. In stage 2, among the remaining weights, we apply column-based BLS (CBBLS) to further compress remaining weights. Finally, a two-stage sparse accelerator, which integrates the TSSC method and corresponding hardware optimization (hardware-efficient activation selection unit and bit-serial multiply-accumulate), is implemented to accelerate the CNN models. Experimental results demonstrate that our proposed TSSC method can reduce the weights of VGG16 and ResNet50 to 0.99b and 0.95b while maintaining an acceptable accuracy drop. When deployed on FPGA, our accelerator achieves an 8.08× weight compression and 7.13× performance improvement compared with 8b non-sparse VGG16 model. Compared with state-of-the-art accelerators, we achieve 1.95×-7.97×, 2.00×-32.8× and 1.13×-6.53× improvement in LUT efficiency, equivalent LUT efficiency and energy efficiency, respectively.