Reading What a Model Is Doing: Operation-Level Legibility Across Scale, Training, and Architecture

Read full text
Authors: Nathan Ryan Young
Cited by: 5

Abstract

Most interpretability instruments read what a model is representing. This paper builds and validates an instrument that names what the model is DOING: a "function camera" that identifies the running in-context operation (the verb) from activation signatures alone, transfer-proof against unseen vocabulary (1.000) and unseen sequence length (0.979), with a bag-of-tokens cheater at chance. Two doctrines fall out of the instrument's own failure modes and are reported with them: a two-timescale rule (integrated features for sustained operations, point features for events -- the long-exposure pathology was found in our own first camera and fixed under seal), and a completed two-by-two organizing result: training DARKENS a model's contents while it CRYSTALLIZES its actions (operation legibility rises 0.719 to 0.947 across the training run, Spearman 0.90), and scale brightens contents in-family while clarifying actions (0.706 to 0.996, seed-robust). An adversarial camouflage arm (payloads embedded in natural prose, never seen in training) reads at 0.986 with zero prose-confusion and the surface control at chance. Honest scope: the camera is validated on synthetic operation families and one substrate pair; open-set discovery and independent replication are named as owed before any stronger claim. All experiments were pre-registered with sealed gates committed before code was written; negative results and self-caught instrument flaws are banked at full weight in the companion verification record. Author contribution and use of AI: the research program and claims are the author's; experiments, derivations, and drafting were done in collaboration with Claude, an AI system by Anthropic, under the author's direction, who verified the results and is responsible for the work. See the corresponding section in the PDF.

Fields of Study & Subjects

Explainable Artificial Intelligence (XAI) Embodied and Extended Cognition Generative Adversarial Networks and Image Synthesis

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
14 July 2026
Document type
Preprint