Joint Optimization of Dynamic Batching and Adaptive Partitioning for Distributed LLMs Inference in Mobile Edge Computing

Large language models (LLMs) are revolutionizing various fields due to their powerful generation capabilities. However, their immense computational complexity poses significant challenges in resource consumption, inference latency, and data privacy for traditional cloud-centric deployments. Edge artificial intelligence (Edge-AI) offers promising LLMs deployment solutions by leveraging distributed resources at the network edge. However, existing approaches struggle to adapt to dynamic workloads and efficiently utilize heterogeneous resources in Mobile Edge Computing (MEC) environments. This paper proposes aDynamicBatching andAdaptivePartitioning (DyBAP) scheme for LLMs deployment, which utilizes ubiquitous geo-distributed resources via end-edge-cloud collaboration. Firstly, we formulate a collaboration deployment optimization problem to minimize inference latency and resource usage under heterogeneous resource and user requirements for latency and accuracy constraints, which is NP-hard. Secondly, to solve this, we develop a dynamic batch fusion optimization algorithm that optimizes the batch size of inference by utilizing the parallel processing power of computing units to balance the latency and resource usage. A block-aware partition optimization algorithm based on multi-agent reinforcement learning (MARL) is proposed for efficient transformer block allocation, integrating mobility awareness for optimal partitioning across dynamic network environments. Simulation results demonstrate the superiority of DyBAP over other benchmarks, reducing inference latency by 17.94% and saving 11.12% in memory resource consumption compared to the end-edge-cloud collaboration approaches.