The Router is a Ruler: A Zero-Parameter Margin Budget for Mixture-of-Experts Routing Instability, Blind Across Four Labs' Models

Read full text
Authors: Nathan Ryan Young
Cited by: 4

Abstract

A mixture-of-experts router is a thresholded selector: a linear score assigns each token to its top-k experts, and the decision is fragile exactly when the k-th and (k+1)-th scores are close. This paper shows that fragility is calculable in advance, under sealed pre-registration. Two instruments carry it. First, the interference law of the companion program transfers to routers unchanged: a compact activity summary predicts the full interference-versus-threshold curve of a production router blind, on held text, with zero parameters fitted to the evaluation data (OLMoE, 64 experts: raw 8.1–11.8% median relative error, calibrated 0.4–1.0%). Second, and centrally, a closed-form margin budget — a union bound over top-k versus next-k expert pairs, scored by the Gaussian tail of each logit gap over the per-pair expert distance — predicts the per-token routing flip rate under representation noise with no free parameters at all. On four labs' models with nothing refit (in-house 30M twins, AllenAI OLMoE, Alibaba Qwen1.5-MoE, IBM granite-3.0) it calls the flip curve decile-by-decile across up to four orders of magnitude, and flips concentrate in the bottom margin decile; a second, near-orthogonal gauge reads routing indecision. The budget bridges to production quantization (real INT8 within 1.3x; per-channel and weight-side hold; INT4 activations fail exactly as pre-named). Because it sits on the router's own decision quantity, it is designable: a training-time margin penalty yields a durable flip reduction calculated before training finishes — reported with its full history, from a crown result that replicated as a fluke, through a dose-response resolution, to a claim-grade re-replication delivering 14x/34x/90x fewer flips at a real, disclosed 0.14–0.40% cross-entropy cost. Measurement claims stand at 1–14B production models; control claims are on the 30M testbed, scale-up named as future work. No claim here is about what an expert means. Author contribution and use of AI: the research program and claims are the author's; experiments and drafting were done in collaboration with Claude, an AI system by Anthropic, under the author's direction, who verified the results and is responsible for the work. See the corresponding section in the PDF.

Fields of Study & Subjects

Complex Network Analysis Techniques Mobile Crowdsensing and Crowdsourcing Big Data and Digital Economy

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
6 July 2026
Document type
Preprint