Tracking finite-time Lyapunov exponents to robustify neural ODEs

This study explores finite-time Lyapunov exponents (FTLEs) in the context of deep neural networks. We consider prototypical classification tasks carried out via neural ODEs (nODEs), which express the network depth as a continuous time variable in a finite interval. In this context, we demonstrate that FTLEs, which measure the exponential separation of input perturbations, are powerful organizers for the nODE input-output dynamics. They allow for better interpretability and the comparison of structurally different dynamics. We establish a direct connection between Lyapunov exponents and adversarial vulnerability, and propose a novel training algorithm that improves robustness by FTLE regularization. The key idea is to suppress large positive exponents, which enhances robustness compared to standard training. To reduce computational costs, we propose focusing the regularization on the early stage of the input-output dynamics. This avoids a full “double” backpropagation, necessary for the parameter gradients of the FTLE regularization term. We provide numerical experiments for a two-dimensional toy dataset as well as for the MNIST dataset.