The Transactions of the Korean Institute of Electrical Engineers P (전기학회논문지P)

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지

Parameter Learning of Dynamic Bayesian Networks using Constrained Least Square Estimation and Steepest Descent Algorithm

제약조건을 갖는 최소자승 추정기법과 최급강하 알고리즘을 이용한 동적 베이시안 네트워크의 파라미터 학습기법

  • 조현철 (울산과학대학 전가전자학부) ;
  • 이권순 (동아대학교 전기공학과) ;
  • 구경완 (호서대학교 국방과학기술학과)
  • Published : 2009.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents new learning algorithm of dynamic Bayesian networks (DBN) by means of constrained least square (LS) estimation algorithm and gradient descent method. First, we propose constrained LS based parameter estimation for a Markov chain (MC) model given observation data sets. Next, a gradient descent optimization is utilized for online estimation of a hidden Markov model (HMM), which is bi-linearly constructed by adding an observation variable to a MC model. We achieve numerical simulations to prove its reliability and superiority in which a series of non stationary random signal is applied for the DBN models respectively.

Keywords

References

  1. K Murphy, "Dynamic Bayesian networks:Representation, Inference and Leaming" Ph.D. Dissertation, UC Berkeley, 2002
  2. X. Li, M. Parizeau, and R. Plamondon, "Training hidden Markov models with multiple observations - A combinatorial method," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.22, no.4, pp. 371-377, 2000 https://doi.org/10.1109/34.845379
  3. M. Stone, "Cross-validatory choice and assenssment of statistical predictions", J. of the Royal Statistical Society. vol. B36, no.2, pp. 111-133, 1974
  4. J. M. Pena, J. Bjorkegren, and J. Tebner, "Learning dynamic Bayesian network models via cross-validation," Pattern Recognition Letters, vol. 26, no.14, pp. 2295-2308. 2005 https://doi.org/10.1016/j.patrec.2005.04.005
  5. R. E. Schapire, Bartlett, P. Y. Freund, and W. S. Lee, "Boosting the margin: a new explanation for the effectiveness of voting methods," Annals of Statistics, vol. 26, no.5, pp.1651-1686, 1998 https://doi.org/10.1214/aos/1024691352
  6. A. Garg, V. Pavlovic, and J. M.Rehg, "Boosted learning in dynamics Bayesian networks for multimodal speaker detection," Proc. of the IEEE, vol. 91, no.9, pp.1355-1369, 2003 https://doi.org/10.1109/JPROC.2003.817119
  7. F. Tian, H. Zhang, and Y. Lu, "Research on modeling with dynamics Bayesian networks," Proc. of the IEEE/WIC Int. Conf. on Web Intelligene, pp. 606-609, 2003
  8. J. A. Bilmes, "Buride Markov models: a graphical-modeling approach to automatic speech recognition," Computer Speech and Language, vol. 17, pp. 213-231, 2003 https://doi.org/10.1016/S0885-2308(03)00010-X
  9. L. Cohen and A. Bronstein, "Adaptive online learning of Bayesian network parameters," http://www.hpl.hp.com/techreports/2001/HPL-2001-156.pdf, 2001
  10. J.-T. Chien, "Online unsupervised learning of hidden Markov models for adaptive speech recognition," IEEE Proc. of Vision, Image and Signal Processing, vol. 148, no.5, pp. 315-324, 2001 https://doi.org/10.1049/ip-vis:20010560
  11. A Papoulis, S. U. Pillai, Probability, random variables and stochastic processes, McGraw Hill, 2002
  12. V. Krishnamurthy, J.B.Moore, and S. -H.Chung, "Hidden Markov model signal processing in presence of unknown deterministic inferences." IEEE Trans. on Automatic Control, vol. 38, no.1, pp. 146-152, 1993 https://doi.org/10.1109/9.186328