WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching

LLM inference is increasingly initiated at edge devices but still executed in centralized GPU clusters. Distributed speculative edge serving offloads drafting to devices, yet existing systems scale poorly under heterogeneous, latency-sensitive workloads. We identify two bottlenecks: Wasted Drafting Time (WDT), where devices over-draft tokens later rejected by the verifier, and Verification Interference, where long verifications delay short requests within a micro-batch. We present WISP, a lossless distributed speculative serving system that combines dynamic drafting with SLO-aware verification batching. WISP uses a lightweight rejection predictor, a batch-latency estimator, and an interference-aware scheduler. Across simulation and prototype experiments, WISP improves system capacity by up to 2.1× over centralized serving and 4.1× over SLED, and improves system goodput by up to 1.94× and 3.7×, while reducing token-speed SLO violations under bursty multi-tenant workloads.