Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Numerical Iteration for Stationary Probabilities of Markov Chains

  • Na, Seongryong (Department of Information and Statistics, Yonsei University)
  • Received : 2014.08.19
  • Accepted : 2014.11.11
  • Published : 2014.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

We study numerical methods to obtain the stationary probabilities of continuous-time Markov chains whose embedded chains are periodic. The power method is applied to the balance equations of the periodic embedded Markov chains. The power method can have the convergence speed of exponential rate that is ambiguous in its application to original continuous-time Markov chains since the embedded chains are discrete-time processes. An illustrative example is presented to investigate the numerical iteration of this paper. A numerical study shows that a rapid and stable solution for stationary probabilities can be achieved regardless of periodicity and initial conditions.

Keywords

References

  1. Na, S. (2010). Markov modeling of multiclass loss systems, The Korean Journal of Applied Statistics, 23, 747-757. https://doi.org/10.5351/KJAS.2010.23.4.747
  2. Nesterov, Y. and Nemirovski, A. (2014). Finding the stationary states of Markov chains by iterative methods, Applied Mathematics and Computation, Available from: http:/www.sciencedirect.com/science/article/pii/s0096300314005931, In press.
  3. O'Leary, D. P. (1993). Iterative methods for finding the stationary vector for Markov chains, Linear Algebra, Markov Chains, and Queueing Models (C. D. Meyer and R. J. Plemmons (ed.)), The IMA volumes in mathematics and its applications, 48, 125-136, Springer, New York.
  4. Ross, S. M. (1996). Stochastic Processes, 2nd edition, Wiley, New York.
  5. Stewart, W. J. (2000). Numerical methods for computing stationary distributions of finite irreducible Markov chains, Computational Probability (W. K. Grassmann (ed.)), International series in operations research and management science, 24, 81-111, Springer, New York. https://doi.org/10.1007/978-1-4757-4828-4_4
  6. Zhao, D., Li, H. and Su, D. (2012). A numerical algorithm on the computation of the stationary distribution of a discrete time homogeneous finite Markov chain, Mathematical Problems in Engineering, 2012, Article ID 167453, 10 pages.

Cited by

  1. Multiclass loss systems with several server allocation methods vol.29, pp.4, 2016, https://doi.org/10.5351/KJAS.2016.29.4.679