Journal of Institute of Control, Robotics and Systems (제어로봇시스템학회논문지)

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Bayesian Recurrent Neural Network for Time Series Prediction

시계열 자료의 예측을 위한 베이지안 순환 신경망에 관한 연구

  • Published : 2004.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, the Bayesian recurrent neural network is proposed to predict time series data. A neural network predictor requests proper learning strategy to adjust the network weights, and one needs to prepare for non-linear and non-stationary evolution of network weights. The Bayesian neural network in this paper estimates not the single set of weights but the probability distributions of weights. In other words, the weights vector is set as a state vector of state space method, and its probability distributions are estimated in accordance with the particle filtering process. This approach makes it possible to obtain more exact estimation of the weights. In the aspect of network architecture, it is known that the recurrent feedback structure is superior to the feedforward structure for the problem of time series prediction. Therefore, the recurrent neural network with Bayesian inference, what we call Bayesian recurrent neural network (BRNN), is expected to show higher performance than the normal neural network. To verify the proposed method, the time series data are numerically generated and various kinds of neural network predictor are applied on it in order to be compared. As a result, feedback structure and Bayesian learning are better than feedforward structure and backpropagation learning, respectively. Consequently, it is verified that the Bayesian reccurent neural network shows better a prediction result than the common Bayesian neural network.

Keywords

References

  1. 김해경, 김태수, 시계열 분석과 예측 이론, 경문사, 2003
  2. 이덕기, 예측 방법의 이해, 고려정보산업, 1999
  3. C. Chatfield, Time-series forecasting, Chapman & Hall/ CRC, 2000
  4. P. J. Brockwell and R. A. Davis, Introduction to time series and forecasting, Springer, New York, 2002
  5. R. G. Brown and P. Y. C. Hwang, Introduction to random signals and applied Kalman filter, John Wiley & Sons, 1997
  6. G. Kitagawa, 'Non-gaussian state-space modeling of nonstationary time series', Journal of the American Statistical Association, vol. 82, no. 400, pp. 1032-1036, 1987 https://doi.org/10.2307/2289375
  7. G. C. Franco and R. C. Souza, 'A comparison of methods for bootstrapping in the local level model', Journal of Forecasting, 21, pp. 27-38, 2002 https://doi.org/10.1002/for.814
  8. O. A. Alsayegh, 'Annual energy consumption prediction using particle filters', Seventh International Symposium on Signal Processing and Its Applications, vol. 2, pp. 571-574, 2003 https://doi.org/10.1109/ISSPA.2003.1224941
  9. A. C. Tsakoumis, S. S. Vladov and V. M. Mladenov, 'Electric load forecasting with multilayer perceptron and elman neural network', 2002 6th Seminar on Neural Network Applications in Electrical Engineering, pp. 87-90, 2002
  10. J. Zhang, K. S. Tang and K. F. Man, 'Recurrent NN model for chaotic time series prediction', 23rd International Conference on Industrial Electronics, Control and Instrumentation, vol. 3, pp. 1108-1112, 1997 https://doi.org/10.1109/IECON.1997.668440
  11. M. Crucianu, 'Bayesian learning for recurrent neural networks', Neurocomputing, 36, pp. 235-242, 2001 https://doi.org/10.1016/S0925-2312(00)00331-3
  12. C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995
  13. J. Durbin and S. J. Koopman, Time Series Analysis by State Space Methods, Oxford University Press, 2001
  14. N. J. Gordon, D. J. Salmond and A. F. M. Smith, 'Novel approach to nonlinear/non-Gaussian bayesian state estimation', Radar and Signal Processing, vol. 140, Issue 2, pp. 107-113, 1993 https://doi.org/10.1049/ip-f-2.1993.0015
  15. T. L. Song, 'Filtering theory', Journal of Control, Automation, and Systems Engineering, vol. 9, no. 6, pp. 413-419, 2003 https://doi.org/10.5302/J.ICROS.2003.9.6.413
  16. D. Pena, G. C. Tiao and R. S. Tsay, A course in time series analysis, John Wiley & Sons, 2001
  17. V. Kadirkamanathan and M. Niranjan, 'A function estimation approach to sequential learning with neural networks', Neural Computation, vol. 5, pp. 954-975, 1993 https://doi.org/10.1162/neco.1993.5.6.954
  18. D. J. C. Mackay, 'A practical bayesian framework for backpropagation networks', Neural Computation, vol. 4, no. 3, pp. 448-472, 1992 https://doi.org/10.1162/neco.1992.4.3.448
  19. R. M. Neal, Bayesian learning for neural network, Lecture Notes in Statistics No. 118, Springer-Verlag, New York, 1996
  20. N. Bergman, Recursive Bayesian estimation : navigation and tracking applications, Linkoping University, 1999
  21. J. F. D. Freitas, M. Niranjan and A. H. Gee, 'Hybrid sequential Monte carlo/kalman methods to train neural networks in non-stationary environments', IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 1057-1060, 1999 https://doi.org/10.1109/ICASSP.1999.759925
  22. T. Zhang and A. Fukushige, 'Forecasting time series by bayesian neural networks', Proceedings of the 2002 International Joint Conference on Neural Networks, vol. 1, pp. 382-387, 2002 https://doi.org/10.1109/IJCNN.2002.1005502
  23. E. H. Tito, G. Zaverucha, M. Vellasco and M. Pacheco, 'Bayesian neural networks for electric load forecasting', 6th International Conference on Neural Information Processing, vol. 1, pp. 407 - 411, 1999 https://doi.org/10.1109/ICONIP.1999.844023
  24. A. Vehtari and J. Lampinen, 'Bayesian neural networks for industrial applications', Proceedings of the 1999 IEEE Midnight-Sun Workshop on Soft Computing Methods in Industrial Applications, pp. 63-68, 1999 https://doi.org/10.1109/SMCIA.1999.782709
  25. M. Crucianu, R. Bone and J.-P. A. De Beauville, 'Bayesian learning for time series prediction with exogenous variables', International Joint Conference on Neural Networks, vol. 4, pp. 2594-2599, 1999 https://doi.org/10.1109/IJCNN.1999.833484

Cited by

  1. Chaff Echo Detecting and Removing Method using Naive Bayesian Network vol.19, pp.10, 2013, https://doi.org/10.5302/J.ICROS.2013.13.8009