One-Shot Training for Reduction of Performance Variance in LLM Rotation Quantization

View at publisher Summarise with VIRAT
Authors: Sangki Park, Chanhoon Kim, Ki-Seok Chung
Cited by: 0
References: 11

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Rotation-based quantization techniques have emerged as effective methods for compressing Large Language Models by mitigating the impact of activation outliers. However, existing approaches face significant challenges: random rotation methods like QuaRot exhibit substantial performance variability across different random seeds, while learned rotation methods like SpinQuant require hundreds of calibration samples, increasing the training cost. To address these limitations, we propose OneRot, a novel one-shot rotation-based quantization technique that achieves stable performance with only a single calibration sample. OneRot introduces random perturbations to layer activations during training, which amplify gradients and enable effective rotation matrix learning in a single attempt while simulating diverse activation distributions. Experimental results on Llama-2 7B, Llama-3.2 3B, and Llama-3.2 1B models with 4-bit quantization demonstrate that OneRot reduces perplexity by$0.25,2.88$, and 21.48, respectively, compared to QuaRot, and reduces standard deviation by$0.01,0.24$, and 1.41, respectively, while achieving comparable performance to SpinQuant using 800 times fewer samples. OneRot provides a practical solution for efficient LLM deployment by combining the stability of learned rotations with the computational efficiency of random approaches.

Fields of Study & Subjects

Advanced Neural Network Applications Topic Modeling Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
International Conference on Artificial Intelligence in Information and Communication (ICAIIC)
Pages
173–176
Publication date
24 February 2026
Language
EN