Journal of the Korean Society for Marine Environment & Energy (한국해양환경ㆍ에너지학회지)

The Korean Society for Marine Environment and Energy (한국해양환경•에너지학회)
권호 표지
DOI QR코드

DOI QR Code

Skill Assessments for Evaluating the Performance of the Hydrodynamic Model

해수유동모델 검증을 위한 오차평가방법 비교 연구

  • Kim, Tae-Yun (Pacific Northwest National Laboratory) ;
  • Yoon, Han-Sam (Research Center for Ocean Industrial Development, Pukyong National Univ.)
  • Received : 2011.03.15
  • Accepted : 2011.05.13
  • Published : 2011.05.25
Download PDF
⟨ Previous Next ⟩

Abstract

To evaluate the performance of the hydrodynamic model, we introduced 10 skill assessments that are assorted by two groups: quantitative skill assessments (Absolute Average Error or AAE, Root Mean Squared Error or RMSE, Relative Absolute Average Error or RAAE, Percentage Model Error or PME) and qualitative skill assessments (Correlation Coefficient or CC, Reliability Index or RI, Index of Agreement or IA, Modeling Efficiency or MEF, Cost Function or CF, Coefficient of Residual Mass or CRM). These skill assessments were applied and calculated to evaluate the hydrodynamic modeling at one of Florida estuaries for water level, current, and salinity as comparing measured and simulated values. We found that AAE, RMSE, RAAE, CC, IA, MEF, CF, and CRM are suitable for the error assessment of water level and current, and AAE, RMSE, RAAE, PME, CC, RI, IA, CF, and CRM are good at the salinity error assessment. Quantitative and qualitative skill assessments showed the similar trend in terms of the classification for good and bad performance of model. Furthermore, this paper suggested the criteria of the "good" model performance for water level, current, and salinity. The criteria are RAAE < 10%, CC > 0.95, IA > 0.98, MEF > 0.93, CF < 0.21 for water level, RAAE < 20%, CC > 0.7, IA > 0.8, MEF > 0.5, CF < 0.5 for current, and RAAE < 10%, PME < 10%, CC > 0.9, RI < 1.15, CF < 0.1 for salinity.

해수유동모델의 검증 및 평가를 위해 적용되는 또는 적용가능한 10종류의 모델 오차평가방법 - 네가지의 정량적 평가방법(절대평균오차, 평균제곱근 오차, 상대적 절대평균오차, 백분율모델오차)과 여섯가지의 정성적 평가방법(상관계수, 신뢰지수, 일치지수, 모델효율성, 비용함수, 잔여량계수) - 을 소개하고, 실제 조위, 유속, 염분관측치와 3차원 곡선형 모델(CH3D)에서 구해진 플로리다 하구에서의 수치해에 이들 모델 오차평가방법들을 적용하였다. 조위 및 유속평가시 절대평균오차, 평균제곱근 오차, 상대적 절대평균오차, 상관계수, 일치지수, 모델효율성, 비용함수, 잔여량계수 등이 적합하였다. 그리고 염분평가시 절대평균오차, 평균제곱근 오차, 상대적 절대평균오차, 백분율모델오차, 상관계수, 신뢰지수, 비용함수, 잔여량계수 등의 사용이 타당하였다. 정량/정성적 평가방법들이 서로 유사한 평가경향을 보여 줌으로써, 상호간의 신뢰성도 보여 주었다. 다양한 모델 오차평가방법을 통하여 계산된 평가값을 토대로, 본 연구에서는 조위, 유속, 염분이 잘 재현된 해수유동모델의 평가범위를 제시하였다. 조위의 경우 상대적 절대평균 오차는 10%이내, 상관계수는 0.95이상, 일치지수는 0.98이상, 모델효율성은 0.93이상, 비용함수는 0.21이내이며, 유속의 경우 상대적 절대평균오차는 20%이내, 상관계수는 0.7이상, 일치지수는 0.8이상, 모델효율성은 0.5이상, 비용 함수는 0.5이내이며, 염분의 경우 상대적 절대평균오차와 백분율모델오차는 10%이내, 상관계수는 0.9이상, 신뢰지수는 1.15이내, 비용함수는 0.1이내 이다.

Keywords

References

  1. Haidvogel, D.B., Arango, H., Budgell, W.P., Cornuelle, B.D., Curchitser, E., Lorenzo, E.D., Fennel, K., Geyer, W.R., Hermann, A.J., Laneorlle, L., Levin, J., McWilliams, j.C., Miller, A.J., Moore, A.M., Powell, T.M., Shchepetkin, A.F., Sherwood, C.R., Signell, R.P., Warner, J.C., and Wilkin, J., 2008, "Ocean forecasting in terrain-following coordinates: Formulation and skill assessment of the Regional Ocean Modeling System", Journal of computational physics, No.227, 3595-3624. https://doi.org/10.1016/j.jcp.2007.06.016
  2. Jolliff, J.K., Kindle, J.C., Shulman, I., Penta, B., Friedrichs, M.A.M, Helber, R., and Arnone, R.A., 2009, "Summary diagrams for coupled hydrodynamic-ecosystem model skill assessment", Journal of Marine Systems, No.76, 64-82. https://doi.org/10.1016/j.jmarsys.2008.05.014
  3. Kim, T., 2007, Modeling of Florida's estuaries: upper Charlotte Harbor and Indian River Lagoon, Ph.D. Dissertation, University of Florida, USA, 305pp.
  4. Kim, T., Sheng, Y.P., Park, K., 2010, "Modeling water quality and hypoxia dynamics in Upper Charlotte Harbor, Florida, U.S.A. during 2000", Estuarine, Coastal and Shelf Science, No.90, 250- 263.
  5. Leggett, R.W. and Williams, L.R., 1981, "A reliability index for models", Ecological Modelling, No.13, 303-312. https://doi.org/10.1016/0304-3800(81)90034-X
  6. Loague, K. and Green, R.E., 1991, "Statistical and graphical methods for evaluating solute transport models: Overview and application", Journal of Contaminant Hydrology, No.7, 51-73. https://doi.org/10.1016/0169-7722(91)90038-3
  7. Ma, G., Shi, F., Liu, S. and Qi, D., 2011, "Hydrodynamic modeling of Changjiang Estuary: Model skill assessment and large-scale structure impacts", Applied Ocean Research, No.33, 69-78. https://doi.org/10.1016/j.apor.2010.10.004
  8. Marechal, D., 2004, A soil-based approach to rainfall-runoff modeling in ungauged catchments for England and Wales, PhD Thesis, Cranfield University, 157pp.
  9. Moreels, E., De Neve, S., Hofman, G., and Van Meirvenne, M., 2003, "Simulating nitrate leaching in bare fallow soils: a model comparison", Nutrient Cycling in Agroecosystems, No.67, 137- 144. https://doi.org/10.1023/A:1025526802717
  10. OSPAR Commission, 1998, Report of the modeling workshop on eutrophication issues. 5-8 November 1996, Den Haag, The Netherlands, OSPAR report, 86pp.
  11. Radach, G. and Moll, A., 2006, "Review of three-dimensional ecological modelling related to the North Sea shelf system. Part II: model validation and data needs", Oceanog. Mar. Biol., No.44, 1-60.
  12. Warner, J.C., Geyer, W.R., and Lerczak, J.A., 2005, "Numerical modeling of an estuary: A comprehensive skill assessment", Journal of geophysical research, 110, C05001, doi:10.1029/ 2004JC002691.
  13. Wilmott, C.J., 1982, "Some comments on the evaluation of model performance", Bulletin American Meteorological Society, No. 63, 1309-1313. https://doi.org/10.1175/1520-0477(1982)063<1309:SCOTEO>2.0.CO;2
  14. Zhang, A., Fan, W., and Ji, F., 2007, "The standards for skill assessment of operational marine forecast system", Chinese Journal of Oceanology and Limnology, No.25, 27-35. https://doi.org/10.1007/s00343-007-0027-7

Cited by

  1. Bias Correction of RCP-based Future Extreme Precipitation using a Quantile Mapping Method ; for 20-Weather Stations of South Korea vol.54, pp.6, 2012, https://doi.org/10.5389/KSAE.2012.54.6.133
  2. Local Winds Effects on the Water Surface Variation at the Shallow Estuary, Mobile Bay vol.36, pp.8, 2014, https://doi.org/10.4491/KSEE.2014.36.8.570
  3. Analysis of Scenarios for Environmental Instream Flow Considering Water Quality in Saemangeum Watershed vol.38, pp.3, 2016, https://doi.org/10.4491/KSEE.2016.38.3.117