Scheduling Training-Inference Co-Location in Demand Response for Sustainable Edge AI

In the pursuit of data privacy and reduced latency, the adoption of edge intelligence has surged. Meanwhile, the enormous increase in AI has resulted in significant energy consumption. Edge intelligence plays a crucial role in Energy Demand Response (EDR). However, existing edge intelligence falls short of meeting the demands of co-locating training and inference tasks while satisfying EDR. Specifically, the intertwinement between balancing energy consumption, system delay and model accuracy, and uncertain future inputs adds to the challenge of designing an online sustainable system for co-located training and inference tasks. To address these challenges, we propose a novel two-timescale system for co-locating training and inference EDR. Our approach satisfies EDR by strategically planning training schedules on macro-timescales and migrating inference requests between heterogeneous edges on micro-timescales while minimizing long-term cost. We introduce a novel online polynomial time algorithm that first breaks down the problem into two subproblems, which are subsequently solved using an online-learning-based fractional algorithm and a randomized roun ding algorithm, respectively. Rigorous analysis demonstrates that our approach achieves both sublinear dynamic regret and sublinear dynamic fit. Extensive trace-driven evaluations validate the practical superiority of our approach over multiple existing methods, highlighting its effectiveness in real-world scenarios.