Transformers as a Boundary Case for Contraction-Based Hierarchical Compression

Read full text
Authors: Michael Hanners
Cited by: 3

Abstract

This paper applies the Hierarchical Compression (HC) framework—referred to as Harmonic Coherence in the foundational publications—to transformer language models. Transformer architectures are evaluated against the HC regularity conditions R1–R4, and the contraction-based formulation of HC is tested through a six-formulation admissibility study. The central finding is that transformers constitute a boundary case for contraction-based HC: under the tested bridge, the dynamics are generically expansive rather than contractive, with a transformer-derived damping coefficient α ≈ 0.72 that cannot serve as a universal anchor. Rather than disqualifying the framework, this result demonstrates the importance of the bridge-assumption layer (B1–B3) for correctly mapping HC variables to domain observables. Key contributions: • Regularity-condition audit mapping R1–R4 to transformer architecture properties • Six-formulation contraction admissibility study across 700 transformer layers • Identification of transformer dynamics as generically expansive under contraction-based HC • Layer-wise entropy analysis via logit lens confirming selective concentration • Condition C verification for sub-Shannon compression at transformer fixed points Companion documents: • CER Theorem: 10.5281/zenodo.18668434 • Reconciliation: 10.5281/zenodo.18671909

Fields of Study & Subjects

Natural Language Processing Techniques Speech and dialogue systems Speech Recognition and Synthesis

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
12 March 2026
Document type
Preprint