Nonlinear system identification using a recurrent network in a Bayesian framework

Modern deep neural networks are being widelyexploited to solve challenging learning tasks, including nonlinearsystem identification. Bayesian system identification intrinsicallyencapsulate uncertainty in model parameters and provides fore-casting distribution enabling enhanced analysis, simulation andcontrol system design. Nevertheless, the application of the full Bayesian approach to articulated models as deep neural networksresults quite challenging in practice. In this work we propose anidentification technique for nonlinear dynamic systems exploitinga deep recurrent neural network with Long-Short Term Memory(LSTM) units retaining a Bayesian framework. To such an aim,we stacked the recurrent neural network with a probabilisticlayer, decomposing the nonlinear dynamic model into a combi-nation of flexible functions. Hence, deterministic and stochasticlayers are trained jointly, forcing the learning algorithm totransform the input data sequences into a deterministic featurespace encoded by the LSTM, useful for predictions. Besides, wedeployed a scalable technique based on Variational Inference todeal with the exact inference intractability. We show the effec-tiveness of the proposed approach by the application to a widelyexploited open benchmark for nonlinear system identification