• 응용통계연구
• /
• 제30권1호
• /
• pp.1-12
• /
• 2017

많은 확률과정이 Markov 특성을 만족하거나 근사적으로 만족하는 것으로 가정된다. Markov 과정에서 특히 관심을 끄는 것은 최초통과시간이다. 최초통과시간에 대한 연구는 Wald의 축차분석에서 시작하여 근사적 특성에 대한 많은 연구가 되어왔고 컴퓨터의 발달로 통계계산적 방법이 사용되면서 근사적 결과가 참값에 가까운 값을 계산할 수 있게 되었다. 이 논문은 Markov 과정의 예로서 지수가중 이동평균 관리도를 사용할 때 평균런길이를 계산하는 과정과 계산상의 주의점, 문제점 등을 연구하였다. 이 결과는 다른 모든 Markov 과정에 적용될 수 있으며 특히 Markov 연쇄로의 근사는 확률과정의 특성의 연구에 유용하고 계산적 접근을 용이하게 한다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제7권5호
• /
• pp.1036-1057
• /
• 2013

Wireless body area network (WBAN) is a promising candidate for future health monitoring system. Nevertheless, the path to mature solutions is still facing a lot of challenges that need to be overcome. Energy efficient scheduling is one of these challenges given the scarcity of available energy of biosensors and the lack of portability. Therefore, researchers from academia, industry and health sectors are working together to realize practical solutions for these challenges. The main difficulty in WBAN is the uncertainty in the state of the monitored system. Intelligent learning approaches such as a Markov Decision Process (MDP) were proposed to tackle this issue. A Markov Decision Process (MDP) is a form of Markov Chain in which the transition matrix depends on the action taken by the decision maker (agent) at each time step. The agent receives a reward, which depends on the action and the state. The goal is to find a function, called a policy, which specifies which action to take in each state, so as to maximize some utility functions (e.g., the mean or expected discounted sum) of the sequence of rewards. A partially Observable Markov Decision Processes (POMDP) is a generalization of Markov decision processes that allows for the incomplete information regarding the state of the system. In this case, the state is not visible to the agent. This has many applications in operations research and artificial intelligence. Due to incomplete knowledge of the system, this uncertainty makes formulating and solving POMDP models mathematically complex and computationally expensive. Limited progress has been made in terms of applying POMPD to real applications. In this paper, we surveyed the existing methods and algorithms for solving POMDP in the general domain and in particular in Wireless body area network (WBAN). In addition, the papers discussed recent real implementation of POMDP on practical problems of WBAN. We believe that this work will provide valuable insights for the newcomers who would like to pursue related research in the domain of WBAN.

• 대한수학회지
• /
• 제38권1호
• /
• pp.25-35
• /
• 2001

In this paper we consider an age dependent branching process whose particles move according to a Markov process with continuous state space. The Markov process is assumed to the stationary with independent increments and positive recurrent. We find some sufficient conditions for he Markov motion process such that the empirical distribution of the positions converges to the limiting distribution of the motion process.

• 대한수학회지
• /
• 제61권4호
• /
• pp.693-711
• /
• 2024

A Markov-modulated Bernoulli process is a generalization of a Bernoulli process in which the success probability evolves over time according to a Markov chain. It has been widely applied in various disciplines for modeling and analysis of systems in random environments. This paper focuses on providing analytical characterizations of the Markovmodulated Bernoulli process by introducing key metrics, including success period, failure period, and cycle. We derive expressions for the distributions and the moments of these metrics in terms of the model parameters.

• 대한수학회보
• /
• 제49권1호
• /
• pp.63-74
• /
• 2012

Based on the Hilbert $C^*$-module structure we study the reconstruction theorem for stationary monotone quantum Markov processes from quantum dynamical semigroups. We prove that the quantum stochastic monotone process constructed from a covariant quantum dynamical semigroup is again covariant in the strong sense.

• 대한전기학회논문지:시스템및제어부문D
• /
• 제52권9호
• /
• pp.533-539
• /
• 2003

Reinforcement learning (RL) has been widely used as a learning mechanism of an artificial life system. However, RL usually suffers from slow convergence to the optimum state-action sequence or a sequence of stimulus-response (SR) behaviors, and may not correctly work in non-Markov processes. In this paper, first, to cope with slow-convergence problem, if some state-action pairs are considered as disturbance for optimum sequence, then they no to be eliminated in long-term memory (LTM), where such disturbances are found by a shortest path-finding algorithm. This process is shown to let the system get an enhanced learning speed. Second, to partly solve a non-Markov problem, if a stimulus is frequently met in a searching-process, then the stimulus will be classified as a sequential percept for a non-Markov hidden state. And thus, a correct behavior for a non-Markov hidden state can be learned as in a Markov environment. To show the validity of our proposed learning technologies, several simulation result j will be illustrated.

• International Journal of Control, Automation, and Systems
• /
• 제1권3호
• /
• pp.358-367
• /
• 2003

We consider discrete-time factorial Markov Decision Processes (MDPs) in multiple decision-makers environment for infinite horizon average reward criterion with a general joint reward structure but a factorial joint state transition structure. We introduce the "localization" concept that a global MDP is localized for each agent such that each agent needs to consider a local MDP defined only with its own state and action spaces. Based on that, we present a gradient-ascent like iterative distributed algorithm that converges to a local optimal solution of the global MDP. The solution is an autonomous joint policy in that each agent's decision is based on only its local state.cal state.

• 산업공학
• /
• 제21권1호
• /
• pp.85-95
• /
• 2008

Reliability prediction serves many purposes during the life of a system, so several methods have been developed to predict the parts and systems reliability. MIL-HDBK-217F, among the those methods, has been widely used as a requisite tool for the reliability prediction which is applied to nuclear power plants and their safety regulations. This paper presents the reliability prediction for the DSP(Digital Signal Processor) module composed of three assemblies. One of the assemblies has a monitoring and self test function which is used to enhance the module reliability. The reliability of each assembly is predicted by MIL-HDBK-217F. Based on these predicted values, Markov modelling is finally used to predict the module reliability. Relax 7.7 software of Relax software corporation is used because it has many part libraries and easily handles Markov processes modelling.

• 대한수학회논문집
• /
• 제12권4호
• /
• pp.1033-1038
• /
• 1997

A functional central limit theorem for a class of nonlinear stationary Markov processes which are geometrically Harris ergodic is derived.

• 대한수학회지
• /
• 제33권4호
• /
• pp.983-992
• /
• 1996

Let p(x, dy) be a transition probability function on $(S, \rho)$, where S is a complete separable metric space. Then a Markov process $X_n$ which has p(x, dy) as its transition probability may be generated by random iterations of the form $X_{n+1} = f(X_n, \varepsilon_{n+1})$, where $\varepsilon_n$ is a sequence of independent and identically distributed random variables (See, e.g., Kifer(1986), Bhattacharya and Waymire(1990)).