ODE-based learning on manifold and its expressive power

Abstract Ordinary differential equation (ODE)-based neural networks have attracted increasing attention because of their practical utility and intriguing mathematical properties. Despite these advantages, several fundamental questions remain open. In particular, the issue of expressive power, in terms of the extent to which ODE-based neural networks (NNs) can exactly realize continuous maps, warrants further investigation. Motivated by these challenges, we first formulate two fundamental problems: addressing the expressive capacity and focusing on the learnability of ODE-based neural networks. We then present several cases in which we successfully resolve the learnability problem. Our results offer new theoretical insights and suggest promising directions for enhancing the performance of differential equation-based neural networks.