• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 1994년도 Proceedings of the Korea Automatic Control Conference, 9th (KACC) ; Taejeon, Korea; 17-20 Oct. 1994
• /
• pp.156-161
• /
• 1994

In this paper, we consider an optimal control problem of a nonlinear stochastic system. Dynamic programming approach is employed for the formulation of a stochastic optimal control problem. As an optimality condition, dynamic programming equation so called the Bellman equation is obtained, which seldom yields an analytical solution, even very difficult to solve numerically. We obtain the numerical solution of the Bellman equation using an algorithm based on the finite difference approximation and the contraction mapping method. Optimal controls are constructed through the solution process of the Bellman equation. We also construct a test case in order to investigate the actual performance of the algorithm.

• 한국수자원학회:학술대회논문집
• /
• 한국수자원학회 2006년도 학술발표회 논문집
• /
• pp.354-359
• /
• 2006

The analysis of large-scale water resources systems is often complicated by the presence of multiple reservoirs and diversions, the uncertainty of unregulated inflows and demands, and conflicting objectives. Reinforcement learning is presented herein as a new approach to solving the challenging problem of stochastic optimization of multi-reservoir systems. The Q-Learning method, one of the reinforcement learning algorithms, is used for generating integrated monthly operation rules for the Keum River basin in Korea. The Q-Learning model is evaluated by comparing with implicit stochastic dynamic programming and sampling stochastic dynamic programming approaches. Evaluation of the stochastic basin-wide operational models considered several options relating to the choice of hydrologic state and discount factors as well as various stochastic dynamic programming models. The performance of Q-Learning model outperforms the other models in handling of uncertainty of inflows.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• 제13권1호
• /
• pp.19-30
• /
• 2013

Recently, the constrained index tracking problem, in which the task of trading a set of stocks is performed so as to closely follow an index value under some constraints, has often been considered as an important application domain for control theory. Because this problem can be conveniently viewed and formulated as an optimal decision-making problem in a highly uncertain and stochastic environment, approaches based on stochastic optimal control methods are particularly pertinent. Since stochastic optimal control problems cannot be solved exactly except in very simple cases, approximations are required in most practical problems to obtain good suboptimal policies. In this paper, we present a procedure for finding a suboptimal solution to the constrained index tracking problem based on approximate dynamic programming. Illustrative simulation results show that this procedure works well when applied to a set of real financial market data.

• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• 제19권4호
• /
• pp.417-428
• /
• 2015

Although, in general, the random fluctuation of interest rates gives a limited impact on portfolio optimization, their stochastic nature may exert a significant influence on the process of selecting the proportions of various assets to be held in a given portfolio when the stochastic volatility of risky assets is considered. The stochastic volatility covers a variety of known models to fit in with diverse economic environments. In this paper, an optimal strategy for portfolio selection as well as the smoothness properties of the relevant value function are studied with the dynamic programming method under a market model of both stochastic volatility and stochastic interest rates.

• Industrial Engineering and Management Systems
• /
• 제16권1호
• /
• pp.44-51
• /
• 2017

In today's competitive environment, supply chain management is a major concern for a company. Two of the key issues in supply chain management are transportation and inventory management. To achieve significant savings, companies should integrate these two issues instead of treating them separately. In this paper we develop a framework for modeling stochastic programming in a supply chain that is subject to demand uncertainty. With reasonable assumptions, two stochastic programming models are presented, respectively, including a single-period and a multi-period situations. Our assumptions allow us to capture the stochastic nature of the problem and translate it into a deterministic model. And then, based on the genetic algorithm and stochastic simulation, a solution method is developed to solve the model. Finally, the computational results are provided to demonstrate the effectiveness of our model and algorithm.

• 경영과학
• /
• 제29권1호
• /
• pp.1-14
• /
• 2012

The goal of the multi-reservoir operation planning is to provide an optimal release plan that maximize the reservoir storage and hydropower generation while minimizing the spillages. However, the reservoir operation is difficult due to the uncertainty associated with inflows. In order to consider the uncertain inflows in the reservoir operating problem, we present a Stochastic Dynamic Programming (SDP) model based on the markov decision process (MDP). The objective of the model is to maximize the expected value of the system performance that is the weighted sum of all expected objective values. With the SDP model, multi-reservoir operating rule can be derived, and it also generates the steady state probabilities of reservoir storage and inflow as output. We applied the model to the Geum-river basin in Korea and could generate a multi-reservoir monthly operating plan that can consider the uncertainty of inflow.

• 한국컴퓨터정보학회논문지
• /
• 제21권9호
• /
• pp.91-100
• /
• 2016

In this research, emergency vehicle dispatching problems faced with in the wake of massive natural disasters are considered. Here, the emergency vehicle dispatching problems can be regarded as a single machine stochastic scheduling problems, where the processing times are independently and identically distributed random variables, are considered. The objective of minimizing the expected number of tardy jobs, with distinct job due dates that are independently and arbitrarily distributed random variables, is dealt with. For these problems, optimal static-list policies can be found by solving corresponding assignment problems. However, for the special cases where due dates are exponentially distributed random variables, using a proposed dynamic programming approach is found to be relatively faster than solving the corresponding assignment problems. This so-called Pivot Dynamic Programming approach exploits necessary optimality conditions derived for ordering the jobs partially.

• 한국멀티미디어학회논문지
• /
• 제11권6호
• /
• pp.807-815
• /
• 2008

A simple efficient method of spotting and recognizing hand gestures in video is presented using a network of hidden Markov models and dynamic programming search algorithm. The description starts from designing a set of isolated trajectory models which are stochastic and robust enough to characterize highly variable patterns like human motion, handwriting, and speech. Those models are interconnected to form a single big network termed a spotting network or a spotter that models a continuous stream of gestures and non-gestures as well. The inference over the model is based on dynamic programming. The proposed model is highly efficient and can readily be extended to a variety of recurrent pattern recognition tasks. The test result without any engineering has shown the potential for practical application. At the end of the paper we add some related experimental result that has been obtained using a different model - dynamic Bayesian network - which is also a type of stochastic model.

• Journal of Electrical Engineering and Technology
• /
• 제12권3호
• /
• pp.1064-1072
• /
• 2017

The power of controlled generators in microgrids randomly fluctuate because of the stochastic volatility of the outputs of photovoltaic systems and wind turbines as well as the load demands. To address and dispatch these stochastic factors for daily operations, a dynamic economic dispatch model with the goal of minimizing the generation cost is established via chance-constrained programming. A Monte Carlo simulation combined with particle swarm optimization algorithm is employed to optimize the model. The simulation results show that both the objective function and constraint condition have been tightened and that the operation costs have increased. A higher stability of the system corresponds to the higher operation costs of controlled generators. These operation costs also increase along with the confidence levels for the objective function and constraints.

• Smart Structures and Systems
• /
• 제19권1호
• /
• pp.21-31
• /
• 2017

To control the stochastic vibration of a vibration-sensitive instrument supported on a beam, the beam is designed as a sandwich structure with magneto-rheological visco-elastomer (MRVE) core. The MRVE has dynamic properties such as stiffness and damping adjustable by applied magnetic fields. To achieve better vibration control effectiveness, the optimal bounded parametric control for the MRVE sandwich beam with supported mass under stochastic and deterministic support motion excitations is proposed, and the stochastic and shock vibration suppression capability of the optimally controlled beam with multi-mode coupling is studied. The dynamic behavior of MRVE core is described by the visco-elastic Kelvin-Voigt model with a controllable parameter dependent on applied magnetic fields, and the parameter is considered as an active bounded control. The partial differential equations for horizontal and vertical coupling motions of the sandwich beam are obtained and converted into the multi-mode coupling vibration equations with the bounded nonlinear parametric control according to the Galerkin method. The vibration equations and corresponding performance index construct the optimal bounded parametric control problem. Then the dynamical programming equation for the control problem is derived based on the dynamical programming principle. The optimal bounded parametric control law is obtained by solving the programming equation with the bounded control constraint. The controlled vibration responses of the MRVE sandwich beam under stochastic and shock excitations are obtained by substituting the optimal bounded control into the vibration equations and solving them. The further remarkable vibration suppression capability of the optimal bounded control compared with the passive control and the influence of the control parameters on the stochastic vibration suppression effectiveness are illustrated with numerical results. The proposed optimal bounded parametric control strategy is applicable to smart visco-elastic composite structures under deterministic and stochastic excitations for improving vibration control effectiveness.