Eective Regularisation from Loss-Landscape Geometry: A Unied Derivation of Direction-Dependent Grokking Dynamics

Grokking is the phenomenon in which a neural network memorises its training data and then, after a prolonged delay, suddenly generalises. Two problems remain open: grokking can occur even at weight-decay strength beta=0, and a uniform penalty beta||theta||^2 somehow produces direction-selective compression. We solve a constrained minimisation of F(theta) = L_data(theta) + beta||theta||^2 on the zero-loss manifold M_0 and derive the effective regularisation strength in the Hessian eigendirection v_k: beta_eff(theta, v_k) = beta + max(h_k(theta), 0) / 2, where h_k(theta) is the eigenvalue of the data-loss Hessian. This single formula resolves both open problems and provides interpretations consistent with five independent lines of recent work. Numerical verification on a quadratic surrogate confirms all theoretical predictions to within 0.05% relative error.