Title: An AI Stack: From Scaling AI Workloads to Evaluating LLMs
Abstract
Large language models (LLMs) have taken the world by storm, enabling new applications, intensifying GPU shortages, and raising concerns about the accuracy of their outputs. In this talk, I will present several projects I have worked on to address these challenges. Specifically, I will focus on Ray, a distributed framework for scaling AI workloads, vLLM and SGLang, two high-throughput inference engines for LLMs, and LMArena, a platform for accurate LLM benchmarking. I will conclude with key lessons learned and outline directions for future research
Fields of Study & Subjects
Parallel Computing and Optimization Techniques
Topic Modeling
Natural Language Processing Techniques
Publication details
- Journal / source
- International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)
- Pages
- 3–4
- Publication date
- 10 March 2026
- Language
- EN