Progressive Training of Hybrid Clipped Rectified Linear Unit for Resilient Convolution Neural Networks

Convolutional neural networks used in safety-critical systems must remain reliable under hardware-induced faults. Activation clipping is an effective fault-mitigation technique, yet existing approaches face a fundamental trade-off: layer-wise clipping provides limited resilience, whereas neuron-wise clipping introduces substantial memory overhead. This study proposes a hybrid clipped Rectified Linear Unit activation function that combines both strategies by applying neuron-wise clipping exclusively to the final layer and layer-wise clipping elsewhere. To further improve reliability, we introduce a progressive training strategy that optimizes the clipping thresholds in a layer-by-layer manner using knowledge distillation. This progressive optimization enables tighter threshold selection at each layer, improving fault tolerance while maintaining low memory costs. Experimental results under high error rates show that the proposed approach significantly enhances reliability compared with existing methods, achieving higher accuracy while substantially reducing memory overhead.