Joint Optimization of DNN Model Caching and Request Routing in Mobile Edge Computing

Mobile edge computing (MEC) can pre-cache deep neural networks (DNNs) near end-users, providing low-latency services and improving users’ quality of experience (QoE). However, caching all DNN models at capacity-limited edge servers is difficult, and the impact of model loading time on QoE remains underexplored. We explore dynamic DNNs by disassembling a complete DNN model into interrelated submodels to enable fine-grained joint optimization of submodel caching and request routing to balance inference precision and loading latency. In this paper, we study the joint dynamic model caching and request routing problem in MEC networks, aiming to maximize user request inference precision under constraints of server resources, latency, and model loading time. We propose CoCaR, an offline algorithm based on linear programming and random rounding that optimizes joint decisions with a provable performance bound. Furthermore, we develop an online extension, CoCaROL, to adapt to dynamic and unpredictable request patterns. The simulation results demonstrate that CoCaR improves the average inference precision for user requests by 40.1% over state-of-the-art baselines. In addition, CoCaR-OL achieves an improvement of at least 32.3% in users’ QoE over competitive baselines.