Quantization Methods for Transformer-Based Models on Edge Devices

As the rapid development and the broad usage of Transformer-based models like ChatGPT and Gemini, and the popularity of edge devices like mobile phones, the request of deployment of Transformer-based models on edge devices is growing urgently.However, the original Transformer-based models have numerous parameters and large expense of computation and storage, which made it almost impossible for deployment on edge devices.To tackle this problem, quantization, which is an efficient and energy-saving model compression approach, eventually emerged and gradually developed into the mainstream method of compressing models on edge devices.This article reviews various state-of-the-art quantization methods, focusing on three key areas: image recognition, large language models, and image generation.We analyze their applicability, application requirements, and performance across different tasks, and compare them with typical experiments and case studies.We further identify shortcomings in existing methods in terms of accuracy preservation, model robustness, and hardware adaptability, and propose potential areas for improvement.These findings aim to contribute to a deeper understanding of ondevice model quantization and provide a reference for more efficient and universal large-scale model deployment.