Predictable Failure Regimes and Decode-Time Control in Large Language Models (Cognitive Control Project — Phase One Technical Report)

This technical report documents a systems-level study of degeneration and failure modes in large language models, with a focus on predicting failures from internal representations and mitigating them via decode-time control without retraining or modifying model architectures. We show that repetitive degeneration—a common failure mode in autoregressive generation—is strongly predictable from hidden states before surface repetition occurs. Using a lightweight classifier (~50k parameters) trained on internal activations, we obtain high separation between repeating and non-repeating contexts (up to 125× at the best checkpoint). This demonstrates that repetition corresponds to a distinct internal regime rather than a purely stochastic decoding artifact. Building on this signal, we introduce a risk-conditioned decode-time intervention that selectively applies repetition penalties only when predicted risk is high. This approach leaves the base model’s forward pass unchanged and operates solely at inference time. Across long-form generation evaluations, the method substantially reduces repetition and improves generation stability and lexical diversity. We also report extensive negative results on attention-level and architectural interventions, highlighting training–inference mismatch as a key failure mode for internal control methods. Beyond repetition, the report outlines a general framework for treating other common LLM failure modes (e.g., hedging, verbosity, sycophancy, topic drift) as predictable behavioral regimes amenable to similar monitoring and control. These extensions are discussed as future work and are not claimed as completed results. This document is intentionally scoped as a Phase One technical report. It makes no claims regarding consciousness, agency, subjective experience, or value alignment. The contribution is limited to demonstrating that certain LLM failure modes are anticipatory, detectable, and practically controllable at decode time using lightweight, reversible mechanisms. Code, trained weights, and reproduction details are provided where applicable.