• Proceedings of the Korea Water Resources Association Conference
• /
• 2006.05a
• /
• pp.354-359
• /
• 2006

The analysis of large-scale water resources systems is often complicated by the presence of multiple reservoirs and diversions, the uncertainty of unregulated inflows and demands, and conflicting objectives. Reinforcement learning is presented herein as a new approach to solving the challenging problem of stochastic optimization of multi-reservoir systems. The Q-Learning method, one of the reinforcement learning algorithms, is used for generating integrated monthly operation rules for the Keum River basin in Korea. The Q-Learning model is evaluated by comparing with implicit stochastic dynamic programming and sampling stochastic dynamic programming approaches. Evaluation of the stochastic basin-wide operational models considered several options relating to the choice of hydrologic state and discount factors as well as various stochastic dynamic programming models. The performance of Q-Learning model outperforms the other models in handling of uncertainty of inflows.

• Journal of Korea Water Resources Association
• /
• v.43 no.1
• /
• pp.67-79
• /
• 2010

Korea water resources corporation (K-Water) has developed the real-time water resources management system for the Nakdong and the Geum River basin to efficiently operate multi-purpose dams in the basins. This study has extended to the Han River basin for providing an effective ending target storage of a month to the real-time water resources management system using Sampling Stochastic Dynamic Programming (SSDP), consequently increasing the efficiency of the reservoir system. The optimization model were developed for three reservoirs, named Soyang, Chungju, and Hwacheon, with high priority in terms of the amounts of effective capacity and water supply for the basin. The number of storage state variable for each dam to set an optimization problem has been assigned from the results of sensitivity analysis. Compared with the K-water operating policy with the target water supply elevations, the optimization model suggested in this study showed that the shortfalls are decreased by 37.22 MCM/year for the required water demands in the basin, even increasing 171 GWh in hydro electronic power generation. In addition, the result of a reservoir operating system during the drawdown period applied to real situation demonstrates that additional releases for water quality or hydro electronic power generation would be possible during the drawdown period between 2007 and 2008. On the basis of these simulation results, the applicability of the SSDP model and the reservoir operating system is proved. Therefore, the more efficient reservoir operation can be achieved if the reservoir operating system is extended further to other Korean basins.

• Journal of Korea Water Resources Association
• /
• v.38 no.9 s.158
• /
• pp.737-746
• /
• 2005

Optimization is a process that searches an optimal solution to obtain maximum or minimum value of an objective function. Many researchers have focused on effective search algorithms for the optimum but few researches were interested in establishing the objective function. This study compares two approaches for the objective function: one allows a tradeoff among the objectives and the other does not allow a tradeoff by assigning weights for the absolute priority between the objectives. An optimization model using sampling stochastic dynamic programming was applied to these two objective functions and the resulting optimal policies were compared. As a result, the objective function with no tradeoff provides a decision making process that matches practical reservoir operations than that with a tradeoff allowed. Therefore, it is more reasonable to establish the objective function with no a tradeoff among the objectives for multi-purpose dam operation plan in Korea.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2004.05b
• /
• pp.272-276
• /
• 2004

이수기와 같이 장기적인 관점에서 저수지운영을 해야 하는 관리자는 해당 기간동안의 이익을 최대화하는 전략을 필요로 한다. 이를 위해서는 미래 유입량의 불확실성을 고려한 최적화 모형에 근거한 운영률을 수립해야 할 것이다. 본 연구에서는 금강수계의 이수기를 대상으로 추계학적 최적화 기법인 표본 추계학적 동적계획법(Sampling Stochastic Dynamic Programming)을 적용하여 최적 연계운영 시스템을 개발하였다. 본 연구를 통해 개발된 모형은 상용프로그램인 CSUDP와의 비교를 통해 검증되었으며 이를 기반으로 과거자료를 이용한 SSDP/Hist모형과 앙상블 유량예측(Ensemble Streamflow Prediction)을 이용한 SSDP/ESP모형을 개발하여 두 모형의 장${\cdot}$단점을 비교 분석하였다. 발전부분은 두 모형이 비슷하였으나 용수공급 측면에서는 SSDP/ESP가 SSDP/Hist 보다 우수함을 알 수 있었다.

• Journal of Korea Water Resources Association
• /
• v.39 no.3 s.164
• /
• pp.187-198
• /
• 2006

Korea Water Resources Corporation(KOWACO) has developed the Integrated Real-time Water Management System(IRWMS) that calculates monthly optimal ending target storages by using Sampling Stochastic Dynamic Programming(SSDP) with Ensemble Streamflow Prediction(ESP) running on the $1^{st}$ day of each month. This system, however, has a shortcoming: it cannot reflect the hydrolmeteorologic variations in the middle of the month. To overcome this drawback, in this study updated ESP forecasts three times each month by using the observed precipitation series from the $1^{st}$ day of the month to the forecast day and the historical precipitation ensemble for the remaining days. The improved accuracy and its effect on the reservoir operations were quantified as a result. SSDP/ESP21 that reflects within-a-month hydrolmeteorologic states saves $1\;X\;10^6\;m^3$ in water shortage on average than SSDP/ESP01. In addition, the simulation result demonstrated that the effect of ESP accuracy on the reduction of water shortage became more important when the total runoff was low during the drawdown period.

• Journal of Korea Water Resources Association
• /
• v.56 no.12
• /
• pp.969-980
• /
• 2023

Since the 2000s, ensemble streamflow prediction (ESP) has been actively utilized in South Korea, primarily for hydrological forecasting purposes. Despite its notable success in hydrological forecasting, the original objective of enhancing water resources system management has been relatively overlooked. Consequently, this study aims to demonstrate the utility of ESP in water resources management by creating a simple hypothetical exercise for dam operators and applying it to actual multi-purpose dams in South Korea. The hypothetical exercise showed that even when the means of ESP are identical, different costs can result from varying standard deviations. Subsequently, using sampling stochastic dynamic programming (SSDP) and considering the capacity-inflow ratio (CIR), optimal release patterns were derived for Soyang Dam (CIR = 1.345) and Chungju Dam (CIR = 0.563) based on types W and P. For this analysis, Type W was defined with standard deviation equal to the mean inflow, and Type P with standard deviation ten times of the mean inflow. Simulated operations were conducted from 2020 to 2022 using the derived optimal releases. The results indicate that in the case of Dam Chungju, more aggressive optimal release patterns were derived under types with smaller standard deviations, and the simulated operations demonstrated satisfactory outcomes. Similarly, Soyang Dam exhibited similar results in terms of optimal release, but there was no significant difference in the simulation between types W and P due to its large CIR. Ultimately, this study highlights that even with the same mean values, the standard deviation of ESP impacts optimal release patterns and outcomes in simulation. Additionally, it underscores that systems with smaller CIRs are more sensitive to such uncertainties. Based on these findings, there is potential for improvements in South Korea's current operational practices, which rely solely on single representative values for water resources management.