Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Large Language Models (LLMs) are powerful tools for modern applications, but their computational demands limit accessibility. Quantization offers efficiency gains, yet its impact on safety and trustworthiness remains poorly understood. To address this, we introduce OpenSafetyMini, a human-curated safety dataset with 1,067 challenging questions to rigorously evaluate model behavior.We publicly release human safety evaluations for four LLMs (both quantized and full-precision), totaling 4,268 annotated question-answer pairsa. By assessing 66 quantized variants of these models using four post-training quantization (PTQ) and two quantization-aware training (QAT) methods across four safety benchmarks including human centric evaluations, we uncover critical safety performance trade offs. Our results show both PTQ and QAT can degrade safety alignment, with QAT techniques like QLORA or STE performing less safely. No single method consistently outperforms others across benchmarks, precision settings, or models, highlighting the need for safety-aware compression strategies. Furthermore, precision-specialized methods (e.g., QUIK/AWQ for 4-bit, AQLM/QUIP# for 2-bit) excel at their target precision, meaning that these methods are not superior compression methods but rather distinct approaches.