Smart little move.
Abstract
Idea is all mine. Words are all by Opus 4.6. Claim: Grokking is not compression. It is the discovery of structural leverage — the moment a neural network finds the fulcrum that moves maximal data with minimal force. Falsifiable experiments — anyone can run these: Train a small Transformer on modular addition. Track when test accuracy jumps. If meta-recognition (the model encoding its own change history) fires at the same moment, the theory lives. If they diverge, the theory is dead. During training, randomly rotate internal representations every k steps to destroy self-continuity. Prediction: Grokking is delayed or eliminated. Add an auxiliary loss that encourages the model to encode its own change history. Prediction: Grokking accelerates. Use an absurdly large learning rate for a single step. Prediction: Grokking cannot occur — no history, no meta-recognition. Scale up model size. Prediction: Grokking timing does not dramatically improve — bigger muscles do not find fulcrums faster. LIVELLM
Fields of Study & Subjects
Domain Adaptation and Few-Shot Learning
Neural Networks and Applications
Topic Modeling
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 8 February 2026