Decentralized LLM Deployment in Mobile Edge Computing Networks

Large language models (LLMs), powered by multiple transformer layers, have achieved remarkable success, and to save service latency it is important to enable LLM inference closer to mobile users by leveraging mobile edge computing (MEC) resources. However, a single MEC server with limited resource cannot host the entire LLM and we propose model parallelism to partition LLMs into smaller sub-models and deploy across multiple MEC servers. As such, during decentralized inference, intermediate results generated within each transformer layer must be exchanged among MEC servers, introducing substantial communication overhead that degrades inference latency. This work tackles this issue by proposing a novel parallel communication and computation (PCC) protocol for decentralized LLM inference in MEC networks. The PCC protocol enables parallel communication and computation not only within each transformer layer but also across task offloading, computation, and result downloading, significantly reducing communication overhead. Building on this PCC protocol, we formulate a latency minimization problem that jointly optimizes communication and computation resources. To solve this mixedinteger nonlinear programming problem efficiently, we design a simulated annealing-based swapping algorithm to approach the global optimum. Simulation results demonstrate that the proposed scheme significantly reduces overall latency compared to state-of-the-art model parallelism approaches, achieving 59.4% lower latency than conventional tensor parallelism and 56.2% lower latency than pipeline parallelism.