Solver-Dependent Representation Learning in Third-Order Neural ODEs

Read full text Summarise with VIRAT
Authors: Gavin Lee Goodship
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Neural ODEs are often evaluated in terms of accuracy, efficiency, or optimization stability, while the numerical solver is treated as an implementation detail. This codebase supports experiments showing that, even when validation accuracy is matched within typical run-to-run variability, different solvers can induce substantially different latent trajectory geometries. In particular, the experiments compare RK4 and extended-stability Runge–Kutta (ESRK) solvers, and measure solver-dependent differences in curvature and jerk using a third-order state formulation. The repository is intended to support: training third-order Neural ODE models on CIFAR-10 and CIFAR-100, evaluating solver-dependent geometric statistics, reproducing the reported curvature and jerk comparisons, comparing algebraically equivalent ESRK realizations, reproducing figures and summary tables from the paper. Main contribution of the code The code implements a third-order Neural ODE in which latent position, velocity, and acceleration are part of the model state. This allows curvature and jerk proxies to be computed directly from the learned dynamics without requiring nested differentiation through the solver. The experiments then isolate solver-induced effects by fixing architecture, optimizer, training schedule, and integration horizon while varying the numerical solver.

Fields of Study & Subjects

Numerical Solvers for Ordinary Differential Equations Model Reduction and Neural Networks Neural Networks and Applications

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 January 2026
Language
EN
Document type
Book