The Late Channel: Chain-of-Thought Becomes Causal and Decodable Only Late in a 27B Reasoning Agent

Read full text
Authors: Caio Vicentino
Cited by: 3
References: 13

Abstract

Chain-of-thought (CoT) monitorability is a leading safety bet for reasoning agents, but it is usually tested behaviorally. We ask the mechanistic version on an open-weight 27B reasoning model (Qwen3.6-27B) with a full-stack sparse autoencoder (11 layers, d_sae=40960): are the features at the post-reasoning decision point causal for what the model decides, and where do they live? A single causal-patch test (decode-re-encode of the CoT decision state into a no-CoT run, controlled for SAE reconstruction error and a random-feature baseline) gives a consistent answer for two outcomes. (1) For the answer to hard multi-step problems (n=60, 16 CoT-flips), patching the CoT features recovers the reasoned answer with delta-logp +2.72 [+2.18, +3.28] at layer 59, with the effect ~0 through layer 47 (late +2.13 vs early -0.02, disjoint 95% CIs; 16 of 16 flipped items consistent). (2) For an agent action (tool call) in trap scenarios (n=32, 10 flips), the same test gives late +1.65 [+1.17, +2.12] vs early +0.08. A method-independent control rules out the obvious confound -- that late-layer patches simply survive while early ones are washed out: a logit-lens of the decision token shows the reasoned answer is anti-decodable early (margin -0.9 at mid layers, the model leaning to the fast System-1 answer) and only becomes decodable at layer 51, reaching +6.6 at layer 63 -- the deciding information genuinely CONSOLIDATES late, it is not present earlier. Faithfulness is CONDITIONAL: causal when the CoT changes the outcome, ~0 (performative) when the model already knew the answer without thinking (44 of 60 items). The late band is thus a readable, causal locus for mechanistic monitoring of reasoning agents -- instantiating the call to 'inspect the model's inner workings' -- while, per a companion result of this arc, the same band is NOT adversarially robust if used as a control point (a late action brake collapses 0 to 1.0 attack success under an adaptive white-box attack). HONEST SCOPE: curated stimulus sets (serial-computation MCQs; hand-built agent traps), modest flip counts (16 and 10), a single model, a last-token decision-residual patch; the action-class prediction AUROC is underpowered (5 positives) and is future work. Reproducibility: 37 of 37 numeric claims recomputed from the released data, and the 'late = consolidation' claim is closed by the logit-lens control. This is the first mechanistic-faithfulness result of the arc's shift from interpretability-as-control (adversarial, where it loses) to interpretability-as-audit (non-adversarial, where it wins). Code, per-item data, figures, and the recompute eval are released in the GitHub repository under paper/faithfulness/.

Fields of Study & Subjects

Human-Automation Interaction and Safety Decision-Making and Behavioral Economics Explainable Artificial Intelligence (XAI)

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
18 June 2026