Generative AI as a Systems Problem: Latency, Energy, and Reliability at Scale

The rapid success of large generative AI models has led to a predominantly modelcentric view of artificial intelligence, where innovation is driven primarily by increases in model size, data scale, and training compute. However, as generative AI systems move from laboratory prototypes to real-world deployments, it is increasingly evident that performance, cost, and reliability are constrained not by model design alone, but by the underlying systems that serve them. Latency tail behavior, energy consumption, hardware heterogeneity, network bottlenecks, and operational failures now define the practical limits of large-scale AI. This paper argues that generative AI must be re-framed as a system engineering problem rather than purely a machine learning challenge. We analyze key system-level constraints-tail latency, power and carbon cost, and multi-layer reliability-and show how they dominate user experience and economic feasibility. We further compare cloud and edge deployment trade-offs and propose a research roadmap that integrates hardware, runtime, and model co-design. By adopting a systems-centric perspective, the AI community can build generative systems that are not only intelligent, but also efficient, reliable, and scalable.