The Capacity Knee: Pricing the Working State of a Language Model — A Two-Part Law, and the Break that Measures the Reader

Read full text
Authors: Nathan Ryan Young
Cited by: 2
References: 3

Abstract

How many things can a language model hold in mind at once? Treating the readout-invisible component of pythia-410m's residual stream as a storage channel and superposing n independent operand cargoes in its functional subspace, this paper measures a two-part law. Content is CONSERVED: the summed content-differential survives to n=64 — amplitude superposition without collapse. Delivery is SLOT-BOUNDED: the fraction of cargoes individually recoverable above a null-calibrated threshold breaks at a knee (n*=4 and 6 on two seed lineages), giving a reliable working-state capacity of about TWO items. A derivation committed to the repository BEFORE the experiment predicts the mechanism — signal-to-interference SIR^2(n) = D_eff/(n-1) — and the knee inverts into an instrument: D_eff = 11.1 and 14.3, a dozen effective coordinates, matching the independently measured width of the operand code. A full-amplitude control confirms the break is genuine cross-talk, not amplitude starvation. The scale ladder (pythia 1b to 2.8b) refused its gates honestly and is shipped as the void it is, with two textures at full weight: no ladder capacity bound exceeds ~2 slots, and the read SUPERLINEARIZES with scale. A gentle-injection retry maps the cross-family dose-response and closes that sub-line as instrument-bounded (gentle windows exist at single-cargo level; no dose survives the full ladder). A reader-versus-medium experiment sharpens the attribution: an optimal codebook-equipped external reader breaks at the same knee — the limit lives in the carried code, not the readout — and the READABLE channel externally carries as much working state as the dark one, making wider auditable working memory a write-side engineering target. Sealed branch predictions with gates git-committed before build; misses banked at full weight. Companion verification deposit (including the pre-run derivation document, verbatim): deposit 0l. Methods and drafting in close collaboration with Claude (Anthropic), disclosed throughout; every number traces to a sealed, pre-registered run.

Fields of Study & Subjects

Language and cultural evolution Machine Learning in Materials Science Parallel Computing and Optimization Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
16 July 2026
Document type
Preprint