Delayed random walks: Identification with neural networks

Delayed random walks represent a simple model capturing feedback delay andnoise, relevant for understanding metastable dynamics and stochasticresonance. We construct neural network models that can capture statisticalproperties of delayed random walks as well as generate random sequences withthese statistical properties. We introduce two architectures based on the LongShort-Term Memory (LSTM), a variant of recurrent neural networks: the delayLSTM, which infers the time delay from a time series, and the timeseries LSTM,which forecasts transition probabilities at each timestep. For comparison, wederive a Maximum Likelihood Estimator (MLE) to directly recover the governingparameters (transition probability and delay) from the data. The delay LSTMnot only provides better parameter estimates than the MLE, but is applicableto systems lacking an analytical MLE. The timeseries LSTM is trained on thedelayed random walks and can generate random sequences (surrogate series) thatcorrectly capture statistical properties, such as the root mean squareposition of the original process. The distribution of the generated sequencesshows little deviation from that of the random walks.