FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters

Proceedings of the 21st European Conference on Computer Systems 2026 Open access

Read full text
Authors: Yanying Lin, Shijie Peng, Chengzhi Lu, Chengzhong Xu, Kejiang Ye
DOI: 10.1145/3767295.3769316
Cited by: 4
References: 61

Abstract

Serving Large Language Models (LLMs) in production faces significant challenges from highly variable request patterns and severe resource fragmentation in serverless clusters. Current systems rely on static pipeline configurations that struggle to adapt to dynamic workload conditions, leading to substantial inefficiencies.

Fields of Study & Subjects

Cloud Computing and Resource Management Advanced Data Storage Technologies Distributed and Parallel Computing Systems

Publication details

Journal / source
Proceedings of the 21st European Conference on Computer Systems
Publisher
ACM
Pages
72–88
Publication date
24 April 2026
Language
EN