Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis
Abstract
We introduce a method for identifying which layers in pre-trained autoregressive language models can be frozen, which should be retrained, and which actively interfere with model performance without running a single gradient step. By mapping per-layer activations to complex oscillator phase perturbations in a Kuramoto dynamical system, we compute a cross-layer interference matrix that reveals the global structural relationships between all layers simultaneously. Applied across four models spanning 124M to 7B parameters (GPT-2 Small, Qwen2.5-0.5B, Qwen2.5-1.5B, Mistral-7B), we discover three universal structural properties: (1) early layers 0-4 form a universal constructive foundation across all domains, (2) at 28+ layers, a second constructive pocket emerges in middle layers, and (3) a conflict anchor layer at approximately 60-70% depth maximally interferes with other layers in every model tested. Phase-guided fine-tuning on GPT-2 Small freezing layers 0-3, pruning the conflict anchor layer 6, and retraining only 30% of parameters improved perplexity by 41.7% on multi-domain text while reducing trainable parameters by 70%. The method requires no training to produce layer-level guidance and generalizes across model architectures and scales.
Fields of Study & Subjects
Topic Modeling
Machine Learning in Materials Science
Language and cultural evolution
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 16 June 2026
- Language
- EN
- Document type
- Preprint