Curvature Bifurcation Induced by Self-Consistency Coupling in Neural Loss Landscapes

Read full text
Authors: Moez Abdessattar
Cited by: 0

Abstract

We investigate the geometric effect of adding a self-consistency term of the form ‖f_θ(θ) - θ‖² to a standard task loss in neural networks. Such terms appear increasingly in meta-learning, world models, and reflective architectures, yet their effect on the loss landscape curvature remains poorly understood. We derive the exact Hessian of this augmented loss and show that it decomposes into a positive semidefinite component from linearization and an indefinite component arising from second-order nonlinearities. This indefinite component can induce a curvature bifurcation at a critical weight α_c, where the minimum eigenvalue of the total Hessian crosses zero. Using numerical experiments in high-dimensional settings (n=50-200) with realistic task Hessians and multiple random parameter points, we demonstrate that this phenomenon is robust and reproducible, yielding α_c = 1.85 ± 0.11 under our experimental conditions. The work resolves previously mysterious instabilities in reflective architectures (like the Godelian Collapse) and offers design guidelines for meta-learning and world models.

Fields of Study & Subjects

Neural Networks and Applications Stochastic Gradient Optimization Techniques Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
27 February 2026
Language
EN
Document type
Preprint