HyRNN: Hybrid Recurrent Neural Networks for Approximating Hybrid Dynamical Systems

For a class of hybrid dynamical systems, we show that a re- current neural network with hybrid dynamics, which we refer to as a hybrid dynamic recurrent neural network (HyRNN), can be constructed to approximate solutions to hybrid sys- tems over bounded (hybrid) time horizons. Specifically, given a desired precision level, we show that a hybrid system with dynamics resembling those of recurrent neural networks for continuous-time and discrete-time systems can be designed so that, for each bounded hybrid time horizon, its solutions are close to the solutions to the given hybrid system. Through the use of universal approximation theorems, we show that the approximation result holds for traditional smooth activa- tion functions, such as sigmoid and arctan, and that exten- sions to ReLU functions are possible, and characterize the complexity of the proposed HyRNN.