A 28nm Speculative-Decoding LLM Processor Achieving 105-to-685µs/Token Latency for Billion-Parameter Models
Abstract
LLMs face decoding bottlenecks. Speculative Decoding (SD) reduces latency via a small draft model for serial decoding and a large target model to verify in parallel. Despite this advantage, it still suffers from exponent redundancy, inefficient weights/KV, and hardware underutilization. We propose an SD LLM processor with three innovations: an exponent dual-reuse MAC, draft back-propagation target mixed-precision dataflow, and draft early start parallel compute, achieving$10.29 \times$speed increase.
Fields of Study & Subjects
Parallel Computing and Optimization Techniques
Advanced Neural Network Applications
Low-power high-performance VLSI design
Publication details
- Journal / source
- Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference
- Pages
- 546–548
- Publication date
- 15 February 2026
- Language
- EN