Transformers as a Boundary Case for Contraction-Based Hierarchical Compression
Abstract
This paper applies the Hierarchical Compression (HC) framework—referred to as Harmonic Coherence in the foundational publications—to transformer language models. Transformer architectures are evaluated against the HC regularity conditions R1–R4, and the contraction-based formulation of HC is tested through a six-formulation admissibility study. The central finding is that transformers constitute a boundary case for contraction-based HC: under the tested bridge, the dynamics are generically expansive rather than contractive, with a transformer-derived damping coefficient α ≈ 0.72 that cannot serve as a universal anchor. Rather than disqualifying the framework, this result demonstrates the importance of the bridge-assumption layer (B1–B3) for correctly mapping HC variables to domain observables. Key contributions: • Regularity-condition audit mapping R1–R4 to transformer architecture properties • Six-formulation contraction admissibility study across 700 transformer layers • Identification of transformer dynamics as generically expansive under contraction-based HC • Layer-wise entropy analysis via logit lens confirming selective concentration • Condition C verification for sub-Shannon compression at transformer fixed points Companion documents: • CER Theorem: 10.5281/zenodo.18668434 • Reconciliation: 10.5281/zenodo.18671909
Fields of Study & Subjects
Natural Language Processing Techniques
Speech and dialogue systems
Speech Recognition and Synthesis
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 12 March 2026
- Document type
- Preprint