How to Explain Grokking
Abstract
Abstract Grokking (delayed generalization in learning theory) is a phenomenon in which a system initially memorizes a training sample with overfitting (the absence of generalization property), and then, after a significant period of time, generalization occurs after overfitting. This paper explains grokking by modeling learning using the Langevin dynamics of a stochastic gradient (Brownian motion) and applying ideas of thermodynamics. It is shown that the properties of grokking can be explained as properties of Brownian motion, in particular, the Eyring formula of kinetic theory.
Fields of Study & Subjects
stochastic dynamics and bifurcation
Advanced Thermodynamics and Statistical Mechanics
Diffusion and Search Dynamics
Publication details
- Journal / source
- Computational Mathematics and Mathematical Physics
- Publisher
- Pleiades Publishing Ltd
- Volume
- 66
- Issue
- 5
- Pages
- 953–957
- Publication date
- 1 May 2026
- Language
- EN