Measuring Energy Consumption of LLMs Inferences
Abstract
Large Language Models (LLMs) offer unprecedented language understanding and generation capabilities. However, these advancements come at a cost. LLMs rely heavily on high-performance computing, not only for training but also for inference. This dependence translates into substantial energy consumption, with inference potentially exceeding the already significant environmental impact of training, which can generate thousands of tons of CO2eq. Therefore, quantifying the energy consumption of LLM inference is crucial. This research focuses on measuring this energy use across a wide range of transformer models, from smaller architectures to cutting-edge models like DeepSeek V3/R1. We utilize state-of-the-art inference frameworks and high-end GPUs to ensure accurate assessments.
Fields of Study & Subjects
Natural Language Processing Techniques
Topic Modeling
Machine Learning in Materials Science
Publication details
- Journal / source
- ACM SIGMETRICS Performance Evaluation Review
- Publisher
- Association for Computing Machinery (ACM)
- Volume
- 53
- Issue
- 3
- Pages
- 18–19
- Publication date
- 9 January 2026
- Language
- EN