Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
Abstract
Large language models (LLMs) are increasingly deployed on edge devices. To meet strict resource constraints, real-world deployment has pushed LLM quantization from 8-bit to 4-bit, 2-bit, and now 1.58-bit. Combined with lookup table (LUT)-based inference, CPUs run these ultra-low-bit LLMs even faster than NPUs, opening new opportunities for ubiquitous on-device intelligence.
Fields of Study & Subjects
Advanced Neural Network Applications
Natural Language Processing Techniques
Topic Modeling
Publication details
- Journal / source
- Proceedings of the 24th Annual International Conference on Mobile Systems, Applications and Services
- Publisher
- ACM
- Pages
- 216–230
- Publication date
- 29 May 2026