LMAdam: Enhancing Adam via Linear Multistep Discretization
Abstract
In this paper, we propose a learning algorithm termed linear multistep adaptive moment (LMAdam) to enhance the adaptive moment (Adam) algorithm for machine learning. Considering Adam as a single-step discretization of its continuous counterpart, we develop the LMAdam algorithm based on a linear multistep discretization scheme. We design a feedforward neural network for learning the coefficients of the multistep terms with ensured consistency and select the coefficients to ensure zero stability of the multistep terms. We experimentally demonstrate the superiority of the LMAdam via extensive experimentation on benchmark datasets for training various deep neural networks in three applications.
Fields of Study & Subjects
Machine Learning in Healthcare
Topic Modeling
Machine Learning and ELM
Publication details
- Journal / source
- IEEE/CAA Journal of Automatica Sinica
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Volume
- 13
- Issue
- 1
- Pages
- 161–169
- Publication date
- 1 January 2026