KV Bank: First Validation of KV-State Injection for MoE-YS

Read full text
Authors: Donghui Yang
Cited by: 4

Abstract

First end-to-end validation of KV-State injection for MoE-YS: a model's own routed experts are distilled to canonical text T(e), encoded by the frozen full model into layer-wise static KV banks (executor = full model, before eviction), and injected into the post-eviction resident model via the router's native Top-4 decisions (routing as bank selection). On Qwen1.5-MoE-A2.7B, injection lifts 60-question scores above the full-weight baseline (+0.084 at K=24, +0.167 at K=36), validating the Deem-Yangs / MoE-YS protocol end-to-end. Companion to the FLMI framework and the Deem-Yangs theoretical paper.

Fields of Study & Subjects

Network Traffic and Congestion Control Software System Performance and Reliability Distributed systems and fault tolerance

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
7 September 2026
Document type
Preprint