Phase Transitions in Gradient-Mixed Learning: Path Dependence and Metastability under Label Noise

We study learning dynamics under label noise through a dual-gradient framework that mixes normalized gradients from noisy supervision (structure gradient) and a trusted signal (value gradient). Across more than 1,750 training runs on image classification tasks, we find that the mixing coefficient λ acts as a control parameter inducing an abrupt transition between qualitatively distinct learning regimes. In an ordered regime (moderate λ), training converges stably to low test error, while beyond a boundary, learning collapses into high-error states. Near the transition, we observe hallmarks of first-order-like behavior: a pronounced variance peak, bimodal outcome distributions, and dynamical slowing. Crucially, we find strong path dependence: at η=0.4, ramp-up (λ:0.2→0.6) reaches an ordered-like final state (11.5% ± 0.3% test error) even though it ends at λ=0.6, whereas ramp-down (λ:0.6→0.2) remains trapped at 38.3% ± 2.9% despite ending at λ=0.2, demonstrating strong asymmetric recovery. Controlled interventions that directly set the cosine similarity between gradients demonstrate that gradient-space geometry governs performance (e.g., varying from 22.6% to 11.5% error by manipulating alignment alone). The qualitative phase structure persists across datasets (CIFAR-10/100) and architectures (ResNet, VGG, CNN), though quantitative boundaries shift with model capacity and optimization hyperparameters. These results suggest that noisy learning exhibits phase-transition-like behavior amenable to statistical-physics description, with gradient geometry providing a mechanistic handle for robust optimization.