The Capacity Knee: Pricing the Working State of a Language Model — A Two-Part Law, and the Break that Measures the Reader
Abstract
How many things can a language model hold in mind at once? Treating the readout-invisible component of pythia-410m's residual stream as a storage channel and superposing n independent operand cargoes in its functional subspace, this paper measures a two-part law. Content is CONSERVED: the summed content-differential survives to n=64 — amplitude superposition without collapse. Delivery is SLOT-BOUNDED: the fraction of cargoes individually recoverable above a null-calibrated threshold breaks at a knee (n*=4 and 6 on two seed lineages), giving a reliable working-state capacity of about TWO items. A derivation committed to the repository BEFORE the experiment predicts the mechanism — signal-to-interference SIR^2(n) = D_eff/(n-1) — and the knee inverts into an instrument: D_eff = 11.1 and 14.3, a dozen effective coordinates, matching the independently measured width of the operand code. A full-amplitude control confirms the break is genuine cross-talk, not amplitude starvation. The scale ladder (pythia 1b to 2.8b) refused its gates honestly and is shipped as the void it is, with two textures at full weight: no ladder capacity bound exceeds ~2 slots, and the read SUPERLINEARIZES with scale. A gentle-injection retry maps the cross-family dose-response and closes that sub-line as instrument-bounded (gentle windows exist at single-cargo level; no dose survives the full ladder). A reader-versus-medium experiment sharpens the attribution: an optimal codebook-equipped external reader breaks at the same knee — the limit lives in the carried code, not the readout — and the READABLE channel externally carries as much working state as the dark one, making wider auditable working memory a write-side engineering target. Sealed branch predictions with gates git-committed before build; misses banked at full weight. Companion verification deposit (including the pre-run derivation document, verbatim): deposit 0l. Methods and drafting in close collaboration with Claude (Anthropic), disclosed throughout; every number traces to a sealed, pre-registered run.
Fields of Study & Subjects
Language and cultural evolution
Machine Learning in Materials Science
Parallel Computing and Optimization Techniques
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 16 July 2026
- Document type
- Preprint