A on-line learning algorithm for recurrent neural networks using variational method

변분법을 이용한 재귀신경망의 온라인 학습

In this paper we suggest a general purpose RNN training algorithm which is derived on the optimal control concepts and variational methods. First, learning is regared as an optimal control problem, then using the variational methods we obtain optimal weights which are given by a two-point boundary-value problem. Finally, the modified gradient descent algorithm is applied to RNN for on-line training. This algorithm is intended to be used on learning complex dynamic mappings between time varing I/O data. It is useful for nonlinear control, identification, and signal processing application of RNN because its storage requirement is not high and on-line learning is possible. Simulation results for a nonlinear plant identification are illustrated.