Journal of The Korean Society of Agricultural Engineers (한국농공학회논문집)

The Korean Society of Agricultural Engineers (한국농공학회)
권호 표지
DOI QR코드

DOI QR Code

Impact of Activation Functions on Flood Forecasting Model Based on Artificial Neural Networks

홍수량 예측 인공신경망 모형의 활성화 함수에 따른 영향 분석

  • Kim, Jihye (Department of Rural Systems Engineering, Seoul National University) ;
  • Jun, Sang-Min (Department of Rural Systems Engineering, Seoul National University) ;
  • Hwang, Soonho (Research Institute of Agriculture and Life Sciences, Seoul National University) ;
  • Kim, Hak-Kwan (Graduate School of International Agricultural Technology, Institutes of Green Bio Science and Technology, Seoul National University) ;
  • Heo, Jaemin (Department of Rural Systems Engineering, Seoul National University) ;
  • Kang, Moon-Seong (Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Institutes of Green Bio Science and Technology, Seoul National University)
  • Received : 2020.07.30
  • Accepted : 2020.10.14
  • Published : 2021.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

The objective of this study was to analyze the impact of activation functions on flood forecasting model based on Artificial neural networks (ANNs). The traditional activation functions, the sigmoid and tanh functions, were compared with the functions which have been recently recommended for deep neural networks; the ReLU, leaky ReLU, and ELU functions. The flood forecasting model based on ANNs was designed to predict real-time runoff for 1 to 6-h lead time using the rainfall and runoff data of the past nine hours. The statistical measures such as R2, Nash-Sutcliffe Efficiency (NSE), Root Mean Squared Error (RMSE), the error of peak time (ETp), and the error of peak discharge (EQp) were used to evaluate the model accuracy. The tanh and ELU functions were most accurate with R2=0.97 and RMSE=30.1 (㎥/s) for 1-h lead time and R2=0.56 and RMSE=124.6~124.8 (㎥/s) for 6-h lead time. We also evaluated the learning speed by using the number of epochs that minimizes errors. The sigmoid function had the slowest learning speed due to the 'vanishing gradient problem' and the limited direction of weight update. The learning speed of the ELU function was 1.2 times faster than the tanh function. As a result, the ELU function most effectively improved the accuracy and speed of the ANNs model, so it was determined to be the best activation function for ANNs-based flood forecasting.

Keywords

References

  1. Adamowski, J., and S. O. Prasher, 2012. Comparison of machine learning methods for runoff forecasting in mountainous watersheds with limited data. Journal of Water and Land Development 17(1): 89-97. doi:10.2478/v10025-012-0038-4.
  2. Aggarwal, C. C., 2018. Neural networks and deep learning: Springer.
  3. Chollet, F., 2015. Keras. https://keras.io. Accessed 20Jul. 2020.
  4. Chollet, F., 2017. Deep Learning with Python: Manning.
  5. Clevert, D. A., T. Unterthiner, and S. Hochreiter, 2016. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv preprint. arXiv:1511.07289.
  6. Dawson, C. W., and R. Wilby, 1998. An artificial neural network approach to rainfall-runoff modeling. Hydrological Sciences 43(1): 47-66. doi:10.1080/02626669809492102.
  7. Donate, J. P., C. Paulo, G. S. German, and S. M. Araceli, 2013. Time series forecasting using a weighted crossvalidation evolutionary artificial neural network ensemble. Neurocompution 109: 27-32. doi:10.1016/j.neucom.2012.02.053.
  8. Glorot, X., and Y. Bengio, 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. Thirteenth International Conference on Artificial Intelligence and Statistics: 249-256.
  9. Goodfellow, I., Y. Bengio, and A. Courville, 2016. Deep Learning: MIT press.
  10. Hsu, K. N., H. V. Gupta, and S. sorooshian, 1995. Artificial neural network modeling of the rainfall-runoff process. Water Resources Research 31(10): 2517-2530. doi:10.1029/95WR01955.
  11. Hu, C., Q. Wu, H. Li, S. Jian, N. Li, and Z. Lou, 2018. Deep learning with a long short-term memory networks approach for rainfall-runoff simulation. Water 10(11): 1543. doi:10.3390/w10111543.
  12. Jeung, M., S. Baek, J. Beom, K. H. Cho, Y. Her, and K. Yoon, 2019. Evaluation of random forest and regression tree methods for estimation of mass first flush ratio in urban catchments. Journal of Hydrology 575: 1099-1110. doi:10.1016/j.jhydrol.2019.05.079.
  13. Kang, M. S., and S. W. Park, 2003, Short-term flood forecasting using artificial neural networks. Journal of The Korean Society of Agricultural Engineers 45(2): 45-57 (in Korean).
  14. Kang, M. S., M. G. Kang, S. W. Park, J. J. Lee, and R. H. Yoo, 2006. Application of grey model and artificial neural networks to flood forecasting. Journal of the American Water Resources Association 42(2): 473-486. https://doi.org/10.1111/j.1752-1688.2006.tb03851.x
  15. Kim, J. H., 1993. A study on hydrologic forecasting of stream flows based on artificial neural network. Ph.D. diss., Incheon, Republic of Korea: Inha University (in Korean).
  16. Kingma, D. P., and J. Ba, 2014. Adam: A method for stochastic optimization. arXiv preprint. arXiv:1412.6980.
  17. Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2017. Imagenet classification with deep convolutional neural networks. Communications of the ACM 60(6): 84-90. doi:10.1145/3065386.
  18. Lau, M. M., and K. H. Lim, 2017. Investigation of activation functions in deep belief network. 2017 2nd International Conference on Control and Robotics Engineering (ICCRE): 201-206. doi:10.1109/ICCRE.2017.7935070.
  19. Lee, G. S., S. C. Park, H. M. Lee, and Y. H. Jin, 2000. The prediction of runoff using artificial neural network in the Young-san River. Journal of Korea Water Resources Association 33(1): 251-256 (in Korean).
  20. Marina, C., P. Andreussi, and A. Soldati, 1999. River flood forecasting with a neural network model. Water Resources Research 35(4): 1191-1197. doi:10.1029/1998WR900086.
  21. Ministry of the Interior and Safety, 2018. Statistical yearbook of natural disaster (in Korean).
  22. Moriasi, D. N., M. W. Gitau, N. Pai, and P. Daggupati, 2015. Hydrologic and water quality models: performance measures and evaluation criteria. Transactions of the ASABE 58(6): 1763-1785. doi:10.13031/trans.58.10715.
  23. Sarkar, A., and R. Kumar, 2012. Artificial neural networks for event based rainfall-runoff modeling. Journal of Water Resource and Protection 4(10): 891. doi:10.4236/jwarp.2012.410105.
  24. Solomatine, D., L. M. See, and R. J. Abrahart, 2009. Data-driven modelling: concepts, approaches and experiences. In Practical Hydroinformatics: 17-30.
  25. Song, J. H., Y. Her, K. Suh, M. S. Kang, and H. Kim, 2019. Regionalization of a Rainfall-Runoff Model: Limitations and Potentials. Water 11(11): 2257. doi:10.3390/w11112257.
  26. Xu, B., N. Wang, T. Chen, and M. Li, 2015. Empirical evaluation of rectified activations in convolutional network. arXiv preprint. arXiv:1505.00853.
  27. Young, C. C., W. C. Liu, and M. C. Wu, 2017. A physically based and machine learning hybrid approach for accurate rainfall-runoff modeling during extreme typhoon events. Applied Soft Computing 53: 205-216. doi:10.1016/j.asoc.2016.12.052.
  28. Zealnad, C. M., D. H. Burn, and S. P. Simonovic, 1999. Short term stream flow forecasting using artificial neural networks. Journal of Hydrology 214: 32-48. doi:10.1016/S0022-1694(98)00242-X.