Multi-Token Prediction on a Consumer GPU: A Quality–Throughput Study of Quantized Gemma 4

An independent paired quality-throughput study of QAT Q4_0 Gemma 4 12B IT and 26B-A4B IT multi-token prediction on one NVIDIA RTX 5070 Ti using a pinned CUDA llama.cpp runtime. N=6 was the fastest observed depth in the fixed grid for both models, while exact greedy-output equivalence frequently failed. Paired objective scoring cleared a prospectively frozen -5 percentage-point non-inferiority margin as exploratory pilot evidence. The record contains the paper and a sanitized public reproducibility archive; verbatim benchmark items, raw generated text, complete logs, local paths, model weights, and vendored third-party source are excluded.