Niyama: QoS-Driven High Throughput LLM Inference Serving - ASPLOS 2026 Artifact
Abstract
This is the artifact for the ASPLOS 2026 paper "Niyama: QoS-Driven High Throughput LLM Inference Serving". This archive contains the source code from the niyama_asplos2026 branch of the Sarathi-Serve repository. Original repository: microsoft/sarathi-serve at niyama_asplos2026For setup and reproduction instructions, please refer to the README.md and artifact appendix included in this archive. preprint(arXiv) : [2503.22562] Niyama : Breaking the Silos of LLM Inference Serving
Fields of Study & Subjects
Parallel Computing and Optimization Techniques
Software System Performance and Reliability
Cloud Computing and Resource Management
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 12 January 2026