Physics of generative AI’s atom: Repetition, bias, and beyond

We derive a first-principles physics theory of the individual “atom” at the heart of generative AI such as ChatGPT: the basic attention head. The theory shows how, when, and why its output can become repetitive or suddenly switch to potentially harmful content. In situations where a small subset of attention heads dominates generative AI’s output or multi-layer effects average out, such undesirable microscale behavior will emerge at the macroscale to threaten generative AI’s safety in medical, legal, and business settings. The theory also quantifies the impact of bias from training and fine-tuning. The theory’s 2-spin form suggests why generative AI such as ChatGPT can work so well but hints that a generalized 3-spin attention might be even better. The theory’s similarity to spin bath physics means existing physics expertise could be harnessed to help generative AI become more trustworthy and resilient to manipulation.