Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Comparative assessment of frost event prediction models using logistic regression, random forest, and LSTM networks

로지스틱 회귀, 랜덤포레스트, LSTM 기법을 활용한 서리예측모형 평가

  • Chun, Jong Ahn (Prediction Research Department, Climate Services and Research Division, APEC Climate Center) ;
  • Lee, Hyun-Ju (Climate Analytics Department, Climate Services and Research Division, APEC Climate Center) ;
  • Im, Seul-Hee (Climate Analytics Department, Climate Services and Research Division, APEC Climate Center) ;
  • Kim, Daeha (Department of Civil Engineering, Jeonbuk National University) ;
  • Baek, Sang-Soo (School of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology)
  • 전종안 (APEC 기후센터 기후사업본부 예측기술과) ;
  • 이현주 (APEC 기후센터 기후사업본부 기후분석과) ;
  • 임슬희 (APEC 기후센터 기후사업본부 기후분석과) ;
  • 김대하 (전북대학교, 토목공학과) ;
  • 백상수 (울산과학기술원, 도시환경공학부)
  • Received : 2021.06.08
  • Accepted : 2021.07.04
  • Published : 2021.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

We investigated changes in frost days and frost-free periods and to comparatively assess frost event prediction models developed using logistic regression (LR), random forest (RF), and long short-term memory (LSTM) networks. The meteorological variables for the model development were collected from the Suwon, Cheongju, and Gwangju stations for the period of 1973-2019 for spring (March - May) and fall (September - November). The developed models were then evaluated by Precision, Recall, and f-1 score and graphical evaluation methods such as AUC and reliability diagram. The results showed that significant decreases (significance level of 0.01) in the frequencies of frost days were at the three stations in both spring and fall. Overall, the evaluation metrics showed that the performance of RF was highest, while that of LSTM was lowest. Despite higher AUC values (above 0.9) were found at the three stations, reliability diagrams showed inconsistent reliability. A further study is suggested on the improvement of the predictability of both frost events and the first and last frost days by the frost event prediction models and reliability of the models. It would be beneficial to replicate this study at more stations in other regions.

이 연구의 목적은 서리 발생일과 무상일 기간의 특성을 분석하고 로지스틱 회귀, 랜덤 포레스트, Long-short Term Memory (LSTM) 기법을 활용하여 서리발생 예측모델을 개발하고 평가하는데 있다. 수원, 청주, 광주 지점에서 봄철과 가을철 서리발생 예측모델 개발을 위한 기상변수들을 수집하였으며, 수집기간은 1973년부터 2019년까지이다. 프리시전(precision), 리콜(Recall), f-1 스코어와, AUC 및 Reliability Diagram과 같은 그래피컬 평가기법을 이용해 서리발생 예측모델을 평가하였다. 봄철과 가을철 모두 서리발생일이 줄어드는 경향성(유의수준: 0.01)을 보였다. 0.9 이상의 높은 AUC 값에도 불구하고, 신뢰도는 일정한 값을 보여주지는 않았다. 서리발생일 측뿐만 아니라, 초상일과 종상일을 정확히 예측할 수 있도록 모형 개선이 필요해 보이며, 다른 지역의 더 많은 지점에서 동일한 기법을 적용해 보는 연구가 필요해 보인다.

Keywords

Acknowledgement

This research was supported by the APEC Climate Center.

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jozefowicz, R., Jia, Y., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems, accessed 4 September 2021, .
  2. Bonsal, B.R., Zhang, X., Vincent, L.A., and Hogg, W.D. (2001). "Characteristics of daily and extreme temperatures over Canada." Journal of Climate, Vol. 14, pp. 1959-1976. https://doi.org/10.1175/1520-0442(2001)014<1959:CODAET>2.0.CO;2
  3. Breiman, L. (2001). "Random forests." Machine Learning, Vol. 45, No. 1, pp. 5-32. https://doi.org/10.1023/A:1010933404324
  4. Chevalier, R.F., Hoogenboom, G., McClendon, R.W., and Paz, J.O. (2012). "A web-based fuzzy expert system for frost warnings in horticultural crops." Environmental Modelling and Software, Vol. 35, pp. 84-91. https://doi.org/10.1016/j.envsoft.2012.02.010
  5. Ding, L., Noborio, K., and Shibuya, K. (2019). "Frost forecast using machine learning-from association to causality." Procedia Computer Science, Vol. 159, pp. 1001-1010. https://doi.org/10.1016/j.procs.2019.09.267
  6. Easterling, D.R. (2002). "Recent changes in frost days and the frostfree season in the United States." Bulletin of the American Meteorological Society, Vol. 83, No. 9, pp. 1327-1332. https://doi.org/10.1175/1520-0477(2002)083<1327:RCIFDA>2.3.CO;2
  7. Frich, P., Alexander, L.V., Della-Marta, P., Gleason, B., Haylock, M., Tank, A.K., and Peterson, T. (2002). "Global changes in climatic extremes during the 2nd half of the 20th century." Climate Research, Vol. 19, pp. 193-212. https://doi.org/10.3354/cr019193
  8. Han, J. H., Choi., J.J., Chung, U., Cho, K.S., and Chun, J.P. (2009). "Frostfall forecasting in the Naju pear production area based on discriminant analysis of climatic data." Korean Journal of Agricultural and Forest Meteorology, Vol. 11, No. 4, pp. 135-142. (in Korean with English abstract) https://doi.org/10.5532/KJAFM.2009.11.4.135
  9. Heino, R., Brazdil, R., Forland, E., Tuomenvirta, H., Alexandersson, H., Beniston, M., Pfister, C., Rebetez, M., Rosenhagen, G., Rosner, S., and Wigib, J. (1999). "Progress in the study of climate extremes in northern and central Europe." Climatic Change, Vol. 42, pp. 151-181. https://doi.org/10.1023/A:1005420400462
  10. Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  11. Hosmer Jr, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied logistic regression. John Wiley & Sons. Inc., Hoboken, NJ, U.S., p. 528.
  12. Kalma, J.D., Laughlin, G.P., Caprio, J.M., and Hamer, P.J.C. (1992). The bioclimatology of frost: Its occurrence, impact and protection. Springer, Berlin, Germany, p.144.
  13. Kim, Y., Shim, K.-M., Jung, M.-P., and Choi, I.-T. (2017). "Study on the estimation of frost occurrence classification using machine learning methods." Korean Journal of Agricultural and Forest Meteorology, Vol. 19, No. 3, pp. 86-92. (in Korean with English abstract) https://doi.org/10.5532/KJAFM.2017.19.3.86
  14. Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., and Klein, M. (2002). Logistic regression. Springer, New York, NY, U.S., p. 702.
  15. Kwon, Y.-A., Lee, H.-S. Kown, W.-T., and Boo, K.-O. (2008). "The weather characteristics of frost occurrence days for protecting crops against frost damage." Journal of the Korean Geographical Society, Vol. 43, No. 6, pp. 824-842. (in Korean with English abstract)
  16. Lee, H., Chun, J.A,. Han, H.-H., and Kim, S. (2016). "Prediction of frost occurrences using statistical modeling approaches." Advances in Meteorology, Vol. 2016, Article ID 2075186.
  17. Lee, S. (2005). "Application of logistic regression model and its validation for landslide susceptibility mapping using GIS and remote sensing data." International Journal of Remote Sensing, Vol. 26, No. 7, pp. 1477-1491. https://doi.org/10.1080/01431160412331331012
  18. Naghibi, S.A., Pourghasemi, H.R., and Dixon, B. (2016). "GIS-based groundwater potential mapping using boosted regression tree, classification and regression tree, and random forest machine learning models in Iran." Environmental Monitoring and Assessment, Vol. 188, No. 1, p. 44. https://doi.org/10.1007/s10661-015-5049-6
  19. Ozdemir, A. (2011). "Using a binary logistic regression method and GIS for evaluating and mapping the groundwater spring potential in the Sultan Mountains (Aksehir, Turkey)." Journal of Hydrology, Vol. 405, No. 1-2, pp. 123-136. https://doi.org/10.1016/j.jhydrol.2011.05.015
  20. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). "Scikit-learn: Machine learning in Python." Journal of Machine Learning Research, Vol. 12, pp. 2825-2830.
  21. Rozante, J.R., Gutierrez, E.R., da Silva Dias, P.L., de Almeida Fernandes, A., Alvim, D.S., and Silva, V.M. (2020). "Development of an index for frost prediction: Technique and validation." Meteorological Applications, Vol. 27, No. 1, pp. 1-12.
  22. Sgubin, G., Swingedouw, D., Dayon, G., de Cortazar-Atauri, I.G., Ollat, N., Page, C., and van Leeuwen, C. (2018). "The risk of tardive frost damage in French vineyards in a changing climate." Agricultural and Forest Meteorology, Vol. 250-251, pp. 226-242. https://doi.org/10.1016/j.agrformet.2017.12.253
  23. Snyder, R.L., Paw, U.K.T., and Thompson, J.F. (1987). Passive frost protection of trees and vines, accessed 4 September 2121, .
  24. Temeyer, B.R., Gallus Jr., W.A., Jungbluth, K.A., Burkheimer, D., and McCauley, D. (2003). "Using an Artificial neural network to predict parameters for frost deposition on Iowa Bridgeways." Proceedings of the 2003 Mid-Continent Transportation Research Symposium. Iowa State University, August 21-22, 2003, Ames, IA. accessed 4 September 2021, .
  25. Vitasse, Y., Schneider, L., Rixen, C., Christen, D., and Rebetez, M. (2018). "Increase in the risk of exposure of forest and fruit trees to spring frosts at higher elevations in Switzerland over the last four decades." Agricultural and Forest Meteorology, Vol. 248, pp. 60-69. https://doi.org/10.1016/j.agrformet.2017.09.005
  26. Xiao, L., Liu, L., Asseng, S., Xia, Y., Tang, L, Liu, B., Cao, W., and Zhu, Y. (2018). "Estimating spring frost and its impact on yield across winter wheat in China." Agricultural and Forest Meteorology, Vol. 260-261, pp. 154-164. https://doi.org/10.1016/j.agrformet.2018.06.006
  27. Xue, D., De Baets, B., Van Cleemput, O., Hennessy, C., Berglund, M., and Boeckx, P. (2013). "Classification of nitrate polluting activities through clustering of isotope mixing model outputs." Journal of Environmental Quality, Vol. 42, No. 5, pp. 1486-1497. https://doi.org/10.2134/jeq2012.0456