Exact Dual-Space Optimization for Neural Networks
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
We propose shifting neural network optimization from the parameter space into a dual space of dimensionality. Our core mechanism is the M-matrix—an analytical per-layer Jacobian derived from forward-pass activations, yielding exact gradients without a global computational graph. In this dual formulation, every weight update is the exact optimum of a least-squares problem, solved via a matrix-free Conjugate Gradient (CG) method.
Fields of Study & Subjects
Stochastic Gradient Optimization Techniques
Neural Networks and Applications
Matrix Theory and Algorithms
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 30 March 2026
- Language
- EN
- Document type
- Preprint