Lorentzian Geometry in Loss Landscapes: Phase Transitions, Null Cones, and Symplectic Optimization
Abstract
We report a previously unobserved geometric structure in neural network loss Hessians: zero-curvature (null) directions arising from the real spectrum of H_reg^{-1}J. Individual null residuals reach 10^{-21} in LeNet-5 (61K parameters) and 10^{-15} in a 22-parameter MLP -- verified across training trajectories and multiple architectures. The theoretical foundation is a Spectral Null Cone Theorem: for any 2n-dimensional symmetric matrix H, every real eigenvector of H_reg^{-1}J is H_reg-null (algebraic identity, 14/14 tests). The theorem requires only that the Hessian is indefinite and H_reg^{-1}J has real eigenpairs -- conditions we prove arise from a sharp phase transition in the free energy Hessian (|mu-y| > 1/sqrt(2*pi)) and observe empirically in neural network training (100% of steps for the 22-parameter MLP, 4/4 checkpoints for LeNet-5). We verify that this structure is actionable (boosting null-direction gradients reduces loss by 5-18%), tunable (automated symplectic re-pairing widens the null channel from 19% to 83% of gradient norm), persistent (adaptive re-optimization during training maintains improvement at +2.9% under strict verification, 5/5 seeds), and architecture-dependent (convolutional networks show stronger structure than MLPs of comparable size). 128 numerical tests, 121 passed; 7 failures are documented boundary cases.
Fields of Study & Subjects
Stochastic Gradient Optimization Techniques
Neural Networks and Applications
Machine Learning and Data Classification
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 1 March 2026
- Language
- EN