Assessing the Sustainability of LLM Inference through Energy–Accuracy Analysis

While the performance of large language models (LLMs) continues to improve, the sustainability of inference has become a growing concern due to the high energy demands of large-scale deployments, which contribute to increased carbon emissions and environmental impact. In this paper, we present a systematic empirical evaluation of inference-time energy consumption, and accuracy across 14 open-source LLMs ranging from 6B to 34B parameters, evaluated on eight benchmark datasets spanning four representative task domains: code generation, text summarization, mathematical reasoning, and question answering. Inference-time energy consumption is measured at the GPU level on an NVIDIA A100 (80 GB) GPU, using fine-grained power sampling under controlled and reproducible experimental conditions. Results show that increasing model size often leads to significantly higher energy consumption without proportional improvements in accuracy. In many tasks, mid-sized models achieve comparable or superior performance while requiring substantially less energy. The energy–accuracy trade-off varies across task types, with summarization being the most energy-intensive and question answering the least. By analyzing the energy–accuracy trade-off, this study provides actionable insights for sustainable AI and helps advance energy-aware deployment of LLMs.