Application of the Central Limit Theorem (CLT) for Performance Modeling in AI-Based Inference Systems

Modern AI applications- including deep learning inference services, large-language-model (LLM) serving platforms, and real-time recommendation engines- operate at massive scale and experience high variability in request latency. Individual inference requests exhibit heavily skewed latency distributions due to network jitter, contention, memory stalls, and nondeterministic scheduling effects. However, performance engineers routinely analyze aggregated metrics such as mean latency, batch averages, and windowed system statistics, which display surprisingly stable and Gaussian-like behavior. This paper demonstrates that the Central Limit Theorem (CLT) provides the mathematical basis for this stability. We simulate AI inference latency using heavy-tailed distributions, compute aggregated batch means, and show that the distribution of batch means converges to a normal distribution even when individual latencies are non-normal. The findings confirm that CLT is foundational for AI performance monitoring, AIOps-based anomaly detection, confidence-bound estimation, and large-scale inference stability analysis.