Mathematics of Transformers
Abstract
This chapter develops a first-principles mathematical theory of transformers. Attention is analyzed as an operator, not a heuristic. Boltzmann–Gibbs formulations are central. Normalization is scrutinized rigorously. Tensor-graph representations unify architectures. Transformers are shown to induce hierarchical aggregative games. Training is formalized precisely. Alternative normalizations and difference operators are studied. Mixture-of-experts architectures are analyzed. The chapter concludes with limits on optimality. No single generative architecture is universally best.
Fields of Study & Subjects
Neural Networks and Applications
Mathematics and Applications
Mathematical Analysis and Transform Methods
Publication details
- Journal / source
- Chapman & Hall/CRC eBooks
- Pages
- 49–98
- Publication date
- 6 February 2026
- Language
- EN
- Document type
- Book chapter