Phase Structure of Deterministic Text Generation in GPT-2

We study the phase structure of text generation in the GPT-2 language model. For a fixed prompt, we probe the embedding space by sequentially applying orthogonal two-dimensional rotations to token embedding vectors in different planes of the hidden space. At each step we perform deterministic generation (greedy decoding) and identify the resulting generation regime using a hash of the normalized first line of the output. The resulting phase map reveals that the hidden space of the model is organized into discrete basins of attraction. In the neighborhood of the original embedding state a stable baseline regime dominates, but increasing the magnitude of the phase deformation leads to abrupt transitions into alternative stable regimes. The transition boundaries are strongly anisotropic: in some phase planes the baseline regime persists down to cosine similarity values of approximately 0.87, whereas in others it changes already at values close to 0.9998, corresponding to almost indistinguishable embedding vectors. Further analysis shows that the entropy of the first-token distribution is maximized in the baseline regime and decreases in alternative regimes, indicating increased model confidence within attracting states. The dictionary of hash identifiers captures a finite yet diverse set of stable generation regimes, among which several dominant ones emerge. These results suggest that GPT-2 can be viewed as a dynamical system with a discrete phase landscape, whose basins of attraction possess complex, anisotropic boundaries that do not align with iso-cosine surfaces.