Transactional KV Caching for Speculative Decoding under Paged KV Memory

Speculative decoding accelerates autoregressive generation by verifying multiple draft tokens per targetmodel forward pass while preserving the target distribution [3, 8]. In parallel, modern serving engines rely on paged KV-cache memory (e.g., Page-dAttention/vLLM) to minimize fragmentation and support prefix sharing across requests [7, 16]. However, speculative decoding induces temporary KV growth for uncommitted draft tokens, which interacts poorly with block-granular KV allocation and token-budget schedulers. Production documentation explicitly notes that draft tokens consume extra KV-cache pages and count toward global token limits, limiting observed speedups at higher load [12, 14]. We propose TransKV, a transactional KV-cache abstraction that separates stable committed KV state from a packed speculative KV buffer. Speculative KV writes remain uncommitted until acceptance is known; only the accepted prefix is committed into the paged cache and rejected KV is discarded without rollback. We provide a formal memory and scheduling analysis showing speculative KV pressure reduces from block-sized pages to token-sized buffers. This reduction yields significant concurrency improvements in capacity-limited prefix-shared branching workloads (e.g., up to 1.78× at 𝐵=16, 𝑚=2). On real GPUs (Kaggle Tesla P100 16 GiB and Colab Tesla T4 15 GiB), TransKV increases achievable branch concurrency by up to 1.78× at 𝐵=16, 𝑚=2 and by 1.60× at 𝐵=16, 𝑚=4, with exact output equivalence by construction.