Assessing the Limitations of Activation Clipping for Fault Mitigation in Vision and Language Transformers

Deep Neural Network (DNN) activation clipping is a well-established method for mitigating hardware-induced faults during inference. Clipping constrains activations to predefined ranges and filtering corrupted values, including NaN and Inf. However, high variability in activation distributions, both across and within layers of a model, makes it challenging to define fixed ranges that reliably capture corrupted values beyond extreme cases. In this study, we show that activation clipping alone is insufficient to protect vision and language Transformers from inference faults, and that inputs exhibit markedly different levels of fault sensitivity depending on the model's confidence. Our experiments show that the missclassification rate can reach 10.28% even with per-layer clipping is employed.