Benchmarking the Energy Savings with Speculative Decoding Strategies

arXiv (Cornell University) 2026 Preprint Open access

Read full text
Authors: Rohit Dutta, Paramita Koley, Soham Poddar, Janardan Misra, Sanjay Podder, Naveen Balani, Saptarshi Ghosh, Niloy Ganguly
DOI: 10.48550/arxiv.2602.09113
Cited by: 0

Abstract

Speculative decoding has emerged as an effective method to reduce latency and inference cost of LLM inferences. However, there has been inadequate attention towards the energy requirements of these models. To address this gap, this paper presents a comprehensive survey of energy requirements of speculative decoding strategies, with detailed analysis on how various factors -- model size and family, speculative decoding strategies, and dataset characteristics -- influence the energy optimizations.

Fields of Study & Subjects

Green IT and Sustainability Parallel Computing and Optimization Techniques Topic Modeling

Publication details

Journal / source
arXiv (Cornell University)
Publication date
9 February 2026
Language
EN
Document type
Preprint