Extremely Large Transformers
Abstract
This chapter studies transformers at extreme scale. Mean-field limits are derived rigorously. Self-attention asymptotics are classified. Masked and unmasked regimes are distinguished. Sigmoid and Boltzmann–Gibbs limits are compared. Learning dynamics are analyzed in small-step regimes. Convergence is established through indistinguishability. k-wise state–action limits are introduced. The chapter provides a law-of-large-numbers view of transformers. Scale becomes structure, not approximation.
Fields of Study & Subjects
Topic Modeling
Advanced Neural Network Applications
Domain Adaptation and Few-Shot Learning
Publication details
- Journal / source
- Chapman & Hall/CRC eBooks
- Pages
- 99–112
- Publication date
- 6 February 2026
- Language
- EN
- Document type
- Book chapter