Layer-wise Embedding-Output Dynamics Across LLM Families: Evidence for Phase-Structured Decision Commitment

Read full text
Authors: Davide D'Elia
Cited by: 0

Abstract

We investigate the layer-wise relationship between embedding geometry and output preferences across four diverse LLM families: Pythia-6.9B, Llama-3.1-8B, Apertus-8B (multilingual), and Gemma-2B (SFT). Using a pair-level centroid-asymmetry metric computed across 230 statement pairs with 10,000 bootstrap resamples, we find that embedding-output dynamics are phase-structured: the correlation between embedding clustering and output preference changes sign systematically across network depth. Key Findings:• Late-layer inversion is architecturally robust in larger base models (confirmed in 3/4 tested families).• Decision commitment depth is architecture-dependent (e.g., Llama commits early, Pythia commits late).• Chat templates are associated with a distinct processing regime in instruction-tuned models.• Small SFT models (Gemma-2B) represent a boundary condition where phase structure does not reliably emerge. Methodological Note:This study introduces a pair-level bootstrapping framework (n=230) that resolves Simpson's Paradox artifacts observed in previous category-level aggregations. Related Work:This work continues the research line established in D'Elia (2025): Uniformity Asymmetry (DOI: 10.5281/zenodo.18110161). Repository & Data:https://github.com/buk81/uniformity-asymmetry

Fields of Study & Subjects

Neural and Behavioral Psychology Studies Neurobiology of Language and Bilingualism Mental Health Research Topics

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
4 January 2026
Language
EN
Document type
Preprint