Behavioral Phase Transitions in Small Language Models: Geometric Scaffolding Precedes Behavioral Emergence

Zenodo (CERN European Organization for Nuclear Research) 2026 Preprint Open access

Read full text
Authors: Bryan Sanchez
DOI: 10.5281/zenodo.18865198
Cited by: 0

Abstract

We train GPT-2 class language models from scratch at five scales (7M to 64M parameters) and track eight behavioral dimensions using rho-eval confidence probes. We find that behavioral capabilities emerge through sharp phase transitions, effective dimensionality expansion precedes behavioral emergence by 200-400 training steps, and a non-monotonic scaling anomaly at 64M parameters suggests interference between competing behavioral subspaces.

Fields of Study & Subjects

Topic Modeling Complex Systems and Dynamics Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
4 March 2026
Language
EN
Document type
Preprint