Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Parameter Estimation of Recurrent Neural Networks Using A Unscented Kalman Filter Training Algorithm and Its Applications to Nonlinear Channel Equalization

언센티드 칼만필터 훈련 알고리즘에 의한 순환신경망의 파라미터 추정 및 비선형 채널 등화에의 응용

  • Kwon Oh-Shin (School of Electronic and Information Engineering Kunsan National University)
  • 권오신 (군산대학교 공과대학 전자정보공학부)
  • Published : 2005.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

Recurrent neural networks(RNNs) trained with gradient based such as real time recurrent learning(RTRL) has a drawback of slor convergence rate. This algorithm also needs the derivative calculation which is not trivialized in error back propagation process. In this paper a derivative free Kalman filter, so called the unscented Kalman filter(UKF), for training a fully connected RNN is presented in a state space formulation of the system. A derivative free Kalman filler learning algorithm makes the RNN have fast convergence speed and good tracking performance without the derivative computation. Through experiments of nonlinear channel equalization, performance of the RNNs with a derivative free Kalman filter teaming algorithm is evaluated.

실시간 순환형 훈련 알고리즘(RTRL)과 같이 경사법에 의해 훈련되는 순환형 뉴럴 네트웍(RNN)은 수렴속도가 매우 느린 단점을 지니고 있다. 이 알고리즘은 또한 오차 역전달 처리과정에서 결코 쉽지 않은 미분 계산을 필요로 한다. 본 논문에서는 완전하게 결합된 RNN의 훈련을 위하여 소위 언센티드 칼만필터라고 불리우는 미분없는 칼만필터 훈련 알고리즘을 시스템의 상태공간 상에서 표현하였다. 미분없는 칼만필터 훈련 알고리즘은 순환형 뉴럴 네트웍 훈련시 미분 계산 없이 매우 빠른 수렴속도와 좋은 추정 성능을 보여준다. 비선형 채널 등화 실험을 통하여 미분 없는 칼만필터 훈련 알고리즘을 이용한 RNN의 성능이 향상되었음을 보였다.

Keywords

References

  1. A. F. Atiya and A. G. Parlos, 'New results on recurrent network traning: Unifying the algorithms and accelerating convergence,' IEEE Transactions on Neural Networks, vol.11, pp.697-709, May 2000 https://doi.org/10.1109/72.846741
  2. G. Kechriotis, E. Zervas, and E. S. Manolakos, 'Using recurrent neural networks for adaptive communication channel equalization,' IEEE Transaction on Neural Networks vol, 5, pp. 267-278, March 1994 https://doi.org/10.1109/72.279190
  3. R. Parisi, E. D. Di Claudio, G. Orlandi, and B.D.Rao, 'Fast adaptive digital equalization by recurrent neural networks,' IEEE Transactions on Signal Processing, vol,45, pp. 2731- 2739, November 1997
  4. S. Ong, C. You, S. Choi, and D. Hong, ' A decision feedback recurrent neural equalizer as an infinte impulse response filter,' IEEE Transcation on Signal Processing, vol. 45,pp. 2851 - 2858, November 1997
  5. H. R. Jiang and K. S. Kwak, 'On modified complex recurrent neural network adaptive equalizer,' Journal of Circuits, Systems, and Computers, vol. 11, no. 1, pp. 93-101, 2002 https://doi.org/10.1142/S0218126602000276
  6. B. Hammer and J. J. Steil, 'Tutorial: Perspective on learning with RNNs,' in Proc. of the European Symposium on Artifical Neural Networks(ESANN), pp.357-369, 2002
  7. R J. Williams and D. Ziper, 'A learning algorithms for continually running fully recurrent neural networks,' Neural Computation, vol. 1, pp.270-280, 1989 https://doi.org/10.1162/neco.1989.1.2.270
  8. P. J. Werbos, 'Back-propagation through time: What it does and how to do it,' Proceedings of the IEEE, vol. 78, pp.1550-1560, October 1990
  9. S. Julier, J. Uhlmann, and H. F. Durrant- Whyte, ' A new method for the nonlinear transformation of means and covaroances in filters and estimators,' IEEE Transaction on Auto matic Control, vol. 45, pp. 477-482, March 2000 https://doi.org/10.1109/9.847726
  10. S. J. julier and J. K. Uhlmann, ' A new extension of the Kalman filter to nonlinear systems,' in Proceeding of AeroSence: The 11th International Symposium on Aerospace /Defence Sensing, Simulation and Controls, 1997
  11. S. Haykin, Nerual Networks: a Comprehensive Foundation, 2nd Ed. Upper Saddle River, NJ: Prentice Hall, 1999
  12. E. A. Wan and R van der Merwe, 'The unscented Kalman filter,' in kalman filtering and neural networks, Edited by S. Haykin. John Wiley and Sons, Inc., 2001
  13. E. A. Wan and R van der Merwe, 'The unscented Kalman filter for nonlinear estimation,' in Proceeding of the IEEE 2000 daptiue Systems for Signal Processing, Communications and Control Symposium/As -SPCC), pp. 153-158, 2000
  14. S. Haykin, Adaptive Filter Theory, 4th Ed. Upper Saddle River, NJ Prentice Hall, 2002
  15. S. Chen. B. Mulgrew, and S. McLaughlin, 'Adaptive Bayesian equalizer with decision feedback,' IEEE Transactions on Signal Processing, vol.41, pp. 2918-2927, September 1993
  16. M. Solazzi, A. Uncini, E. D. Di Clauio, and R.Parisi, 'Complex discriminative learning Bayesian neural equalizer,' Signal Processing, vol. 81, pp. 2493-2502, 2001
  17. S. Chen, G. J. Gibson, B. Mulgrew, and S. McLaughlin, 'Adaptive equalization of finite nonliner channels using multilayer perceptrons, ' Signal Processing, vol. 20, pp. 107-119, 1990
  18. C.Cowan and Semnani, 'Time-variant equalization using a novel non-linear adaptive structure,' International Journal of Adaptive Control and Signal Processing, vol. 12, no.2, pp. 195-206, 1998

Cited by

  1. Nonlinear System State Estimating Using Unscented Particle Filters vol.17, pp.6, 2013, https://doi.org/10.6109/jkiice.2013.17.6.1273