Exact Dual-Space Optimization for Neural Networks

Read full text Summarise with VIRAT
Authors: Petrov, Yuriy
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

We propose shifting neural network optimization from the parameter space into a dual space of dimensionality. Our core mechanism is the M-matrix—an analytical per-layer Jacobian derived from forward-pass activations, yielding exact gradients without a global computational graph. In this dual formulation, every weight update is the exact optimum of a least-squares problem, solved via a matrix-free Conjugate Gradient (CG) method.

Fields of Study & Subjects

Stochastic Gradient Optimization Techniques Neural Networks and Applications Matrix Theory and Algorithms

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
30 March 2026
Language
EN
Document type
Preprint