Deterministic Layer Freezing in Autoregressive Language Models via Continuous Phase Coherence Analysis

Read full text
Authors: Nossair Bajddi
Cited by: 4

Abstract

We introduce a method for identifying which layers in pre-trained autoregressive language models can be frozen, which should be retrained, and which actively interfere with model performance without running a single gradient step. By mapping per-layer activations to complex oscillator phase perturbations in a Kuramoto dynamical system, we compute a cross-layer interference matrix that reveals the global structural relationships between all layers simultaneously. Applied across four models spanning 124M to 7B parameters (GPT-2 Small, Qwen2.5-0.5B, Qwen2.5-1.5B, Mistral-7B), we discover three universal structural properties: (1) early layers 0-4 form a universal constructive foundation across all domains, (2) at 28+ layers, a second constructive pocket emerges in middle layers, and (3) a conflict anchor layer at approximately 60-70% depth maximally interferes with other layers in every model tested. Phase-guided fine-tuning on GPT-2 Small freezing layers 0-3, pruning the conflict anchor layer 6, and retraining only 30% of parameters improved perplexity by 41.7% on multi-domain text while reducing trainable parameters by 70%. The method requires no training to produce layer-level guidance and generalizes across model architectures and scales.

Fields of Study & Subjects

Topic Modeling Machine Learning in Materials Science Language and cultural evolution

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
16 June 2026
Language
EN
Document type
Preprint