Smart little move.

Read full text
Authors: SHUHEI CORMIER
Cited by: 0

Abstract

Idea is all mine. Words are all by Opus 4.6. Claim: Grokking is not compression. It is the discovery of structural leverage — the moment a neural network finds the fulcrum that moves maximal data with minimal force. Falsifiable experiments — anyone can run these: Train a small Transformer on modular addition. Track when test accuracy jumps. If meta-recognition (the model encoding its own change history) fires at the same moment, the theory lives. If they diverge, the theory is dead. During training, randomly rotate internal representations every k steps to destroy self-continuity. Prediction: Grokking is delayed or eliminated. Add an auxiliary loss that encourages the model to encode its own change history. Prediction: Grokking accelerates. Use an absurdly large learning rate for a single step. Prediction: Grokking cannot occur — no history, no meta-recognition. Scale up model size. Prediction: Grokking timing does not dramatically improve — bigger muscles do not find fulcrums faster. LIVELLM

Fields of Study & Subjects

Domain Adaptation and Few-Shot Learning Neural Networks and Applications Topic Modeling

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
8 February 2026