• 한국경영과학회지
• /
• 제17권2호
• /
• pp.131-144
• /
• 1992

Even though nonhomogeneous Markov Decision Processes subsume homogeneous Markov Decision Processes and are more practical in the real world, there are many results for them. In this paper we address the nonhomogeneous Markov Decision Process with objective to maximize average reward. By extending works of Ross [17] in the homogeneous case adopting the result of Bean and Smith [3] for the dicounted deterministic problem, we first transform the original problem into the discounted nonhomogeneous Markov Decision Process. Then, secondly, we transform into the discounted deterministic problem. This approach not only shows the interrelationships between various problems but also attacks the solution method of the undiscounted nohomogeneous Markov Decision Process.

• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• 제25권4호
• /
• pp.149-161
• /
• 2021

This study proposes a methodology for mission/path planning of an unmanned aerial vehicle (UAV) using an artificial potential field with the Markov Decision Process (MDP). The planning problem is formulated as an MDP. A low-resolution solution of the MDP is obtained and used to define an artificial potential field, which provides a continuous UAV mission plan. A numerical case study is conducted to demonstrate the validity of the proposed technique.

• 품질경영학회지
• /
• 제11권2호
• /
• pp.2-9
• /
• 1983

As will be seen, this paper is tries that the research on the mathematical structure of Markov process and Markovian sequential decision process (the policy improvement iteration method,) moreover, that it analyze the logic and the characteristic of behavior of mathematical model of Markov process. Therefore firstly, it classify, on research of mathematical structure of Markov process, the forward equation and backward equation of Chapman-kolmogorov equation and of kolmogorov differential equation, and then have survey on logic of equation systems or on the question of uniqueness and existence of solution of the equation. Secondly, it classify, at the Markovian sequential decision process, the case of discrete time parameter and the continuous time parameter, and then it explore the logic system of characteristic of the behavior, the value determination operation and the policy improvement routine.

• 대한산업공학회지
• /
• 제42권1호
• /
• pp.65-72
• /
• 2016

Curling is compared to the Chess because of variety and importance of strategies. For winning the Curling game, selecting optimal strategies at decision making points are important. However, there is lack of research on optimal strategies for Curling. 'Aggressive' and 'Conservative' strategies are common strategies of Curling; nevertheless, even those two strategies have never been studied before. In this study, Markov Decision Process would be applied for Curling strategy analysis. Those two strategies are defined as actions of Markov Decision Process. By solving the model, the optimal strategy could be found at any in-game states.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제7권5호
• /
• pp.1036-1057
• /
• 2013

Wireless body area network (WBAN) is a promising candidate for future health monitoring system. Nevertheless, the path to mature solutions is still facing a lot of challenges that need to be overcome. Energy efficient scheduling is one of these challenges given the scarcity of available energy of biosensors and the lack of portability. Therefore, researchers from academia, industry and health sectors are working together to realize practical solutions for these challenges. The main difficulty in WBAN is the uncertainty in the state of the monitored system. Intelligent learning approaches such as a Markov Decision Process (MDP) were proposed to tackle this issue. A Markov Decision Process (MDP) is a form of Markov Chain in which the transition matrix depends on the action taken by the decision maker (agent) at each time step. The agent receives a reward, which depends on the action and the state. The goal is to find a function, called a policy, which specifies which action to take in each state, so as to maximize some utility functions (e.g., the mean or expected discounted sum) of the sequence of rewards. A partially Observable Markov Decision Processes (POMDP) is a generalization of Markov decision processes that allows for the incomplete information regarding the state of the system. In this case, the state is not visible to the agent. This has many applications in operations research and artificial intelligence. Due to incomplete knowledge of the system, this uncertainty makes formulating and solving POMDP models mathematically complex and computationally expensive. Limited progress has been made in terms of applying POMPD to real applications. In this paper, we surveyed the existing methods and algorithms for solving POMDP in the general domain and in particular in Wireless body area network (WBAN). In addition, the papers discussed recent real implementation of POMDP on practical problems of WBAN. We believe that this work will provide valuable insights for the newcomers who would like to pursue related research in the domain of WBAN.

• 인터넷정보학회논문지
• /
• 제10권3호
• /
• pp.85-94
• /
• 2009

본 논문에서는 무선 센서 네트워크에서의 에너지 효율적인 전송방안을 제안하고 이에 대한 이론적 분석을 제시하고자 한다. 본 논문에서 제안하는 전송기법은 채널 상태가 상대적으로 좋을 때만 전송을 시도하는 opportunistic transmission에 기반한 이진 결정 (binary-decision) 기반 전송이다. 이진 결정 기반 전송에서는 Markov decision process (MDP)를 이용하여 성공적인 전송을 위한 최적의 채널 임계값을 도출하였다. 다양한 시뮬레이션을 통해 제안하는 전송기법의 성능을 에너지 효율성과 전송율 측면에서 분석하였다.

• 한국통신학회논문지
• /
• 제36권2A호
• /
• pp.139-148
• /
• 2011

본 논문은 멀티밴드 해양통신망에서 선적 정보를 주기적으로 전송할 때 발생하는 비용을 최소화하기 위해 가용한 네트워크의 전송 비용과 주어진 허용 가능한 최대 지연 범위 이내에서 예상되는 최소 평균 전송 비용을 비교하여 전송 시점을 결정하는 방안을 제시한다. 이때 전송 시점과 해당 네트워크의 선택 과정을 Markov Decision Process (MDP)로 모델링하며, 이에 따라 각 밴드에서의 채널 상태를 2-State Markov Chain으로 모델링하고 평균 전송 비용을 Stochastic Dynamic Programming을 통해 계산한다. 이를 통해 최소 비용의 망 선택 방식이 도출되었으며, 제안된 방식을 사용할 때 고정 주기를 사용하여 정보를 전송하는 방식에 비해 상당한 망 사용 비용을 절감할 수 있음을 컴퓨터 시뮬레이션을 통해 보인다.

• ETRI Journal
• /
• 제29권4호
• /
• pp.527-529
• /
• 2007

In this paper, a decision-tree-based Markov model for phrase break prediction is proposed. The model takes advantage of the non-homogeneous-features-based classification ability of decision tree and temporal break sequence modeling based on the Markov process. For this experiment, a text corpus tagged with parts-of-speech and three break strength levels is prepared and evaluated. The complex feature set, textual conditions, and prior knowledge are utilized; and chunking rules are applied to the search results. The proposed model shows an error reduction rate of about 11.6% compared to the conventional classification model.

• 대한산업공학회지
• /
• 제16권2호
• /
• pp.1-9
• /
• 1990

The control of deterioration processes for which only incomplete state information is available is examined in this study. When the deterioration is governed by a Markov process, such processes are known as Partially Observable Markov Decision Processes (POMDP) which eliminate the assumption that the state or level of deterioration of the system is known exactly. This research investigates a two state partially observable Markov chain in which only deterioration can occur and for which the only actions possible are to replace or to leave alone. The goal of this research is to develop a new jump algorithm which has the potential for solving system problems dealing with continuous state space Markov chains.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제12권5호
• /
• pp.2414-2428
• /
• 2018

In order to solve the problem that the current network security situation assessment methods just focus on the attack behaviors, this paper proposes a kind of network security situation assessment method based on Markov Decision Process and Game theory. The method takes the Markov Game model as the core, and uses the 4 levels data fusion to realize the evaluation of the network security situation. In this process, the Nash equilibrium point of the game is used to determine the impact on the network security. Experiments show that the results of this method are basically consistent with the expert evaluation data. As the method takes full account of the interaction between the attackers and defenders, it is closer to reality, and can accurately assess network security situation.