The Separability Principle of Transfer in Grokking

An empirical study of transfer in the grokking regime: how the structure of source and target tasks affects whether transfer helps or hurts. Across 20+ configurations I find that separable tasks (inputs processed independently) tend to transfer positively (up to ~+88% faster grokking), while interactive tasks (inputs processed jointly) form a distinct family, with cross-family transfer often negative (−49% to −300%, growing with scale). Author's note (revised 2026): This is best read as a grokking-specific instance of the established principle that task similarity/separability governs the sign of transfer — see the deep-transfer task-similarity literature and the analytical positive/negative-transfer threshold for perceptrons (Gerace et al. 2021). The contribution here is the empirical characterization in the grokking setting, not the principle itself. Because grokking is highly sensitive to seeds and hyperparameters, the effect sizes should be treated as preliminary pending replication across seeds.