Deadline-aware service scheduling via multi-head MARL in device-edge-cloud environments

Mobile Edge Computing (MEC) enables latency-sensitive Internet of Things (IoT) applications by offloading computation to nearby edge servers. Yet, most existing intelligent scheduling approaches either neglect the full three-tier device–edge–cloud architecture or fail to account for heterogeneous IoT services composed of multiple dependent tasks with diverse deadlines and resource demands. These gaps hinder adaptability under dynamic network conditions. We propose a multi-agent reinforcement learning (MARL) framework in which Cellular IoT devices and the edge server act as cooperative agents to optimize task offloading and scheduling. Each agent employs a multi-head deep Q-network (MH-DQN)—with one head per service type—to efficiently manage heterogeneous service workflows. We further implement a multi-head Double DQN (MH-DDQN) variant to improve stability and convergence. Simulation results demonstrate that both MH-DQN and MH-DDQN substantially outperform heuristic baselines, sustaining high deadline compliance (up to around 85\% acceptance at heavy load) and lower latency. MH-DDQN achieves the fastest convergence, greater stability, and slightly lower delays, highlighting its advantage for adaptive scheduling in complex MEC environments.