Trained in the Light: Legibility as a Pretraining-Time Choice, Measured at Scale

Read full text
Authors: Nathan Ryan Young
Cited by: 4

Abstract

The alignment between a transformer's residual-stream writes and its unembedding readout is unconstrained by the prediction objective. In paired, seeded, from-scratch pretraining runs (162M and 70M, one architecture family), an unconstrained baseline's mid-layer writes drift to near-chance alignment with the readout span, while a twin trained with one additional loss term concentrates 44% of write energy into the top 8 of 768 readout directions (42x chance; 81% in the top 32), robust to a foreign lens. The price is small and positive on both seeds (matched-step 0.66-1.08%; paired bootstrap +1.15% CI [1.07,1.22]), rising to ~1.4% at 3.5x packing pressure. Reported at equal weight: the concentrated writes are no more token-decodable than baseline (visible is not readable), and unconstrained layers stay near chance. All predictions sealed and git-committed before launch; misses published at full weight; two adversarial review panels run pre-release and their findings incorporated. AI-collaboration disclosure included in the paper.

Fields of Study & Subjects

Advanced Data Storage Technologies Semiconductor materials and devices Parallel Computing and Optimization Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
20 July 2026
Document type
Preprint