One-Shot Training for Reduction of Performance Variance in LLM Rotation Quantization
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Rotation-based quantization techniques have emerged as effective methods for compressing Large Language Models by mitigating the impact of activation outliers. However, existing approaches face significant challenges: random rotation methods like QuaRot exhibit substantial performance variability across different random seeds, while learned rotation methods like SpinQuant require hundreds of calibration samples, increasing the training cost. To address these limitations, we propose OneRot, a novel one-shot rotation-based quantization technique that achieves stable performance with only a single calibration sample. OneRot introduces random perturbations to layer activations during training, which amplify gradients and enable effective rotation matrix learning in a single attempt while simulating diverse activation distributions. Experimental results on Llama-2 7B, Llama-3.2 3B, and Llama-3.2 1B models with 4-bit quantization demonstrate that OneRot reduces perplexity by$0.25,2.88$, and 21.48, respectively, compared to QuaRot, and reduces standard deviation by$0.01,0.24$, and 1.41, respectively, while achieving comparable performance to SpinQuant using 800 times fewer samples. OneRot provides a practical solution for efficient LLM deployment by combining the stability of learned rotations with the computational efficiency of random approaches.
Fields of Study & Subjects
Advanced Neural Network Applications
Topic Modeling
Domain Adaptation and Few-Shot Learning
Publication details
- Journal / source
- International Conference on Artificial Intelligence in Information and Communication (ICAIIC)
- Pages
- 173–176
- Publication date
- 24 February 2026
- Language
- EN