• Journal of the military operations research society of Korea
• /
• v.11 no.1
• /
• pp.111-129
• /
• 1985

Markov chains with large transition probability matrices occur in many applications such as manpowr models. Under certain conditions the state space of a stationary discrete parameter finite Markov chain may be partitioned into subsets, each of which may be treated as a single state of a smaller chain that retains the Markov property. Such a chain is said to be 'lumpable' and the resulting lumped chain is a special case of more general functions of Markov chains. There are several reasons why one might wish to lump. First, there may be analytical benefits, including relative simplicity of the reduced model and development of a new model which inherits known or assumed strong properties of the original model (the Markov property). Second, there may be statistical benefits, such as increased robustness of the smaller chain as well as improved estimates of transition probabilities. Finally, the identification of lumps may provide new insights about the process under investigation.

• Management Science and Financial Engineering
• /
• v.12 no.2
• /
• pp.19-35
• /
• 2006

We use the hierarchical Bayesian approach to describe the transition probabilities of a binary nonhomogeneous Markov chain. The Markov chain is used for describing the transition behavior of emotionally disturbed children in a treatment program. The effects of covariates on transition probabilities are assessed using a logit link function. To describe the time evolution of transition probabilities, we consider two modeling strategies. The first strategy is based on the concept of exchangeabiligy, whereas the second one is based on a first order Markov property. The deviance information criterion (DIC) measure is used to compare models with two different time dependent structures. The inferences are made using the Markov chain Monte Carlo technique. The developed methodology is applied to some real data.

• Journal of Korea Water Resources Association
• /
• v.42 no.3
• /
• pp.213-225
• /
• 2009

A stationary Markov chain model is a stochastic process with the Markov property. Having the Markov property means that, given the present state, future states are independent of the past states. The Markov chain model has been widely used for water resources design as a main tool. A main assumption of the stationary Markov model is that statistical properties remain the same for all times. Hence, the stationary Markov chain model basically can not consider the changes of mean or variance. In this regard, a primary objective of this study is to develop a model which is able to make use of exogenous variables. The regression based link functions are employed to dynamically update model parameters given the exogenous variables, and the model parameters are estimated by canonical correlation analysis. The proposed model is applied to daily rainfall series at Seoul station having 46 years data from 1961 to 2006. The model shows a capability to reproduce daily and seasonal characteristics simultaneously. Therefore, the proposed model can be used as a short or mid-term prediction tool if elaborate GCM forecasts are used as a predictor. Also, the nonstationary Markov chain model can be applied to climate change studies if GCM based climate change scenarios are provided as inputs.

• Management Science and Financial Engineering
• /
• v.19 no.2
• /
• pp.49-53
• /
• 2013

A Markov-chain Monte Carlo sampling algorithm samples a new point around the latest sample due to the Markov property, which prevents it from sampling from multi-modal distributions since the corresponding chain often fails to search entire support of the target distribution. In this paper, to overcome this problem, mode switching scheme is applied to the conventional MCMC algorithms. The algorithm separates the reducible Markov chain into several mutually exclusive classes and use mode switching scheme to increase mixing rate. Simulation results are given to illustrate the algorithm with promising results.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2016.05a
• /
• pp.425-426
• /
• 2016

In this paper, we proposed an algorithm based on movement prediction using Markov chain in delay tolerant networks(DTNs). The existing prediction algorithms require additional information such as a node's schedule and connectivity between nodes. However, network reliability is lowered when additional information is unknown. To solve this problem, we proposed an algorithm for predicting a movement path of the node by using Markov chain. The proposed algorithm maps speed and direction for a node into state, and predict movement path of the node using transition probability matrix generated by Markov chain. As the result, proposed algorithm show that the proposed algorithms has competitive delivery ratio but with less average latency.

• The Korean Journal of Applied Statistics
• /
• v.24 no.4
• /
• pp.657-676
• /
• 2011

Properties of statistical process control procedures may not be derived analytically in many cases; however, the application of a Markov chain can solve such problems. This article shows how to derive the properties of the process control procedures using the generated Markov chains when the control statistic satisfies the Markov property. Markov chain approaches that appear in the literature (such as the statistical design and economic design of the control chart as well as the variable sampling rate design) are reviewed along with the introduction of research results for application to a new control procedure and reset chart. The joint application of a Markov chain approach and analytical solutions (when available) can guarantee the correct derivation of the properties. A Markov chain approach is recommended over simulation studies due to its precise derivation of properties and short calculation times.

• The Korean Journal of Applied Statistics
• /
• v.34 no.4
• /
• pp.537-556
• /
• 2021

Properties of statistical process control procedures may not be derived analytically in many cases; however, the application of a Markov chain can solve such problems. This article shows how to derive the properties of the process control procedures using the generated Markov chains when the control statistic satisfies the Markov property. Markov chain approaches that appear in the literature (such as the statistical design and economic design of the control chart as well as the variable sampling rate design) are reviewed along with the introduction of research results for application to a new control procedure and reset chart. The joint application of a Markov chain approach and analytical solutions (when available) can guarantee the correct derivation of the properties. A Markov chain approach is recommended over simulation studies due to its precise derivation of properties and short calculation times.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.7 no.4
• /
• pp.285-294
• /
• 2007

In this paper, we investigate a novel online estimation algorithm for dynamic Bayesian network(DBN) parameters, given as conditional probabilities. We sequentially update the parameter adjustment rule based on observation data. We apply our algorithm to two well known representations of DBNs: to a first-order Markov Chain(MC) model and to a Hidden Markov Model(HMM). A sliding window allows efficient adaptive computation in real time. We also examine the stochastic convergence and stability of the learning algorithm.

• International Journal of Quality Innovation
• /
• v.1 no.1
• /
• pp.64-74
• /
• 2000

This paper investigates the issue of performance testing on web browsing environments. Among the typical non-functional characteristics, index of link validity will be deeply explored. A framework to certify link correctness in web site is proposed. All possible navigation paths are first formulated to represent a usage model with the Markov chain property, which is then used to generate test script file statistically. With collecting any existing failure information followed by tracing these testing browsed paths, certification analysis may be performed by applying Markov chain theory. The certification result will yield some significant information such as: test coverage, reliability measure, confidence interval, etc. The proposed mechanism may provide not only completed but also systemic methodologies to find any linking errors and other web technologies errors. Besides, an actual practice of the proposed approach to a web-based system will be demonstrated quantitatively through a certification tool.

• Journal of applied mathematics & informatics
• /
• v.30 no.3_4
• /
• pp.541-553
• /
• 2012

In this paper, we study some limit properties of the harmonic mean of random transition probability for a second-order nonhomogeneous Markov chain on the generalized gambling system indexed by a tree by constructing a nonnegative martingale. As corollary, we obtain the property of the harmonic mean and the arithmetic mean of random transition probability for a second-order nonhomogeneous Markov chain indexed by a double root tree.