Reading What a Model Is Doing: Operation-Level Legibility Across Scale, Training, and Architecture
Abstract
Most interpretability instruments read what a model is representing. This paper builds and validates an instrument that names what the model is DOING: a "function camera" that identifies the running in-context operation (the verb) from activation signatures alone, transfer-proof against unseen vocabulary (1.000) and unseen sequence length (0.979), with a bag-of-tokens cheater at chance. Two doctrines fall out of the instrument's own failure modes and are reported with them: a two-timescale rule (integrated features for sustained operations, point features for events -- the long-exposure pathology was found in our own first camera and fixed under seal), and a completed two-by-two organizing result: training DARKENS a model's contents while it CRYSTALLIZES its actions (operation legibility rises 0.719 to 0.947 across the training run, Spearman 0.90), and scale brightens contents in-family while clarifying actions (0.706 to 0.996, seed-robust). An adversarial camouflage arm (payloads embedded in natural prose, never seen in training) reads at 0.986 with zero prose-confusion and the surface control at chance. Honest scope: the camera is validated on synthetic operation families and one substrate pair; open-set discovery and independent replication are named as owed before any stronger claim. All experiments were pre-registered with sealed gates committed before code was written; negative results and self-caught instrument flaws are banked at full weight in the companion verification record. Author contribution and use of AI: the research program and claims are the author's; experiments, derivations, and drafting were done in collaboration with Claude, an AI system by Anthropic, under the author's direction, who verified the results and is responsible for the work. See the corresponding section in the PDF.
Fields of Study & Subjects
Explainable Artificial Intelligence (XAI)
Embodied and Extended Cognition
Generative Adversarial Networks and Image Synthesis
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 14 July 2026
- Document type
- Preprint