Journal of Environmental Impact Assessment (환경영향평가)

Korean Society of Environmental Impact Assessment (한국환경영향평가학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Statistical Parameters for the Evaluation of Regional Air Quality Modeling Results - Focused on Fine Dust Modeling -

지역규모 대기질 모델 결과 평가를 위한 통계 검증지표 활용 - 미세먼지 모델링을 중심으로 -

  • Kim, Cheol-Hee (Department of Atmospheric Sciences, Pusan National University) ;
  • Lee, Sang-Hyun (Department of Atmospheric Science, Kongju National University) ;
  • Jang, Min (Research Center for Atmospheric Environment, Hankuk University of Foreign Studies) ;
  • Chun, Sungnam (Korea Electric Power Corporation Research Institute, Korea Electric Power Corporation) ;
  • Kang, Suji (Korea Electric Power Corporation Research Institute, Korea Electric Power Corporation) ;
  • Ko, Kwang-Kun (Institution of East and West Studies, Yonsei University) ;
  • Lee, Jong-Jae (School of Urban and Environmental Engineering, Ulsan National Institute of Science & Technology (UNIST)) ;
  • Lee, Hyo-Jung (Department of Atmospheric Sciences, Pusan National University)
  • 김철희 (부산대학교 대기환경과학과) ;
  • 이상현 (공주대학교 대기과학과) ;
  • 장민 (한국외국어대학교 대기환경연구센터) ;
  • 천성남 (한국전력공사 전력연구원) ;
  • 강수지 (한국전력공사 전력연구원) ;
  • 고광근 (연세대학교 동서문제연구원) ;
  • 이종재 (울산과학기술원 도시환경공학부) ;
  • 이효정 (부산대학교 대기환경과학과)
  • Received : 2020.04.24
  • Accepted : 2020.07.24
  • Published : 2020.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

We investigated statistical evaluation parameters for 3D meteorological and air quality models and selected several quantitative indicator references, and summarized the reference values of the statistical parameters for domestic air quality modeling researcher. The finally selected 9 statistical parameters are MB (Mean Bias), ME (Mean Error), MNB (Mean Normalized Bias Error), MNE (Mean Absolute Gross Error), RMSE (Root Mean Square Error), IOA (Index of Agreement), R (Correlation Coefficient), FE (Fractional Error), FB (Fractional Bias), and the associated reference values are summarized. The results showed that MB and ME have been widely used in evaluating the meteorological model output, and NMB and NME are most frequently used for air quality model results. In addition, discussed are the presentation diagrams such as Soccer Plot, Taylor diagram, and Q-Q (Quantile-Quantile) diagram. The current results from our study is expected to be effectively used as the statistical evaluation parameters suitable for situation in Korea considering various characteristics such as including the mountainous surface areas.

본 연구에서는 3차원 기상 및 대기질 모델의 입출력 자료를 평가하는 데 필요한 통계 검증지표를 선별하고, 선정된 검증지표의 기준치를 조사하여 그 결과를 요약하였다. 여러 국내외 문헌과 최근 논문 검토를 통해 최종 선정된 통계 검증지표는 MB (Mean Bias), ME (Mean Error), MNB (Mean Normalized Bias Error), MNE (Mean Absolute Gross Error), RMSE (Root Mean Square Error), IOA (Index of Agreement), R (Correlation Coefficient), FE (Fractional Error), FB (Fractional Bias)로 총 9가지이며, 국내외 문헌을 통해 그 기준치를 확인하였다. 그 결과, 기상모델의 경우 대부분 MB와 ME가 주요 지표로 사용되어 왔고, 대기질 모델 결과는 NMB와 NME 지표가 주로 사용되었으며, 그 기준치의 차이를 분석하였다. 아울러 이들 통계 검증지표값을 이용하여 모델 예측 결과를 효과적으로 비교하기 위한 표출 도식으로 축구 도식, 테일러 도식, Q-Q (Quantile-Quantile) 도식의 장단점을 분석하였다. 나아가 본 연구 결과를 기반으로 우리나라의 산악지역의 특수성 등이 잘 고려된 통계 검증지표의 기준치 설정 등의 추가연구가 효과적으로 진행될 수 있기를 기대한다.

Keywords

References

  1. Borge R, Alexandrov J, Lumbreras J, Rodriguez E. 2008. A comprehensive sensitivity analysis of the WRF model for air quality applications over the Iberian Peninsula. Atmospheric Environment. 42: 8560-8574. https://doi.org/10.1016/j.atmosenv.2008.08.032
  2. Boylan JW, Russell AG. 2006. PM and light extinction model performance metrics, goals, and criteria for three dimensional air quality models. Atmospheric Environment. 40: 4946-4959. https://doi.org/10.1016/j.atmosenv.2005.09.087
  3. Byun DW, Kim ST, Kim SB. 2007. Evaluation of air quality models for the simulation of a high ozone episode in the Houston metropolitan area. Atmospheric Environment. 41: 837-853. https://doi.org/10.1016/j.atmosenv.2006.08.038
  4. Chen D, Xie X, Zhou Y, Lang J, Xu T, Tang N, Zhao Y, Liu X. 2017. Performance Evaluation of the WRF-Chem Model with Different Physical Parameterization Schemes during an Extremely High $PM_{2.5}$ Pollution Episode in Beijing. Aerosol and Air Quality Research. 17: 262-277. https://doi.org/10.4209/aaqr.2015.10.0610
  5. Choi DR, Koo YS, Jo JS, Jang YK, Lee JB, Park HJ. 2016. The effect of dust emissions on $PM_{10}$ concentration in East Asia, Journal of Korean Society for Atmospheric Environment. 32(1): 32-45. [Korean Literature] https://doi.org/10.5572/KOSAE.2016.32.1.032
  6. Emery C, Tai E, Yarwood G. 2001. Enhanced Meteorological Modeling and Performance Evaluation for Two Texas Ozone Episodes, Prepared for the Texas Natural Resource Conservation Commission. ENVIRON International Corporation. Novato, CA.
  7. Emery C, Liu Z, Russell AG, Odman MT, Yarwood G, Kumar N. 2017. Recommendations on statistics and benchmarks to assess photochemical model performance. Journal of the Air & Waste Management Association 67(5): 582-598. https://doi.org/10.1080/10962247.2016.1265027
  8. Environ. 2014. User's Guide: Comprehensive Air Quality Model with Extensions Version 6.1, http://www.camx.com.
  9. Environ and Alpine. 2012. Western Regional Air Partnership (WRAP) West-wide Jump-start Air Quality Modeling Study (WestJump AQMS) - WRF Application/Evaluation. ENVIRON International Corporation, Novato, California. Alpine Geophysics, LLC. University of North Carolina. February 29.
  10. Fox DG. 1981. Judging air quality modeled performance. Bulletin of American Meteorological Society 62: 599-609. https://doi.org/10.1175/1520-0477(1981)062<0599:JAQMP>2.0.CO;2
  11. Grell GA, Emeis S, Stockwell WR, Schoenemeyer T, Forkel R, Michalakes J, Knoche R, Seidl W. 2000. Application of a multiscale, coupled MM5/chemistry model to the complex terrain of the VOTALP valley campaign. Atmospheric Environment. 34: 1435-1453. https://doi.org/10.1016/S1352-2310(99)00402-1
  12. Grell GA, Peckham SE, Schmitz R, McKeen SA, Frost G, Skamarock WC, Eder B. 2005. Fully coupled "online" chemistry within the WRF model. Atmospheric Environment. 39: 6957-6975. https://doi.org/10.1016/j.atmosenv.2005.04.027
  13. Hogrefe C, Civerolo KL, Hao W, Ku JY, Zalewsky EE, Sistla G. 2008. Rethinking the assessment of photochemical modeling systems in air quality planning applications. J. Air Waste Management Association. 58: 1086-1099. https://doi.org/10.3155/1047-3289.58.8.1086
  14. Hogrefe C, Hao W, Zalewsky EE, Ku JY, Lynn B, Rosenzweig V, Schultz MG, Rast S, Newchurch MJ, Wang L, Kinney PL, Sistla G. 2011. An analysis of long-term regional-scale ozone simulations over the northeastern United States: Variability and trends. Atmospheric Chemistry and Physics 11: 567-582. https://doi.org/10.5194/acp-11-567-2011
  15. Hurley PJ. 1999. The Air Pollution Model (TAPM) Version 1: Technical Description and Examples. CSIRO: Clayton, Australia. 11: 567-582.
  16. Ju H, Bae C, Kim BU, Kim H, Yoo C, Kim S. 2017. $PM_{2.5}$ Source Apportionment Analysis to Investigate Contributions of the Major Source Areas in the Southeastern Region of South Korea. Journal of Korean Society for Atmospheric Environment. 34(4): 517-533. [Korean Literature] https://doi.org/10.5572/KOSAE.2018.34.4.517
  17. Korea Electric Power Corporation Research Institute. 2020. Study of $PM_{2.5}$ in Korea (SPIKE). Contributions from different sources and regions:3002017909.
  18. Kim S, Kim O, Kim BU, Kim HC. 2017. Impact of Emissions from Major Point Sources in Chungcheongnam-do on Surface Fine Particulate Matter Concentration in the Surrounding Area. Journal of Korean Society for Atmospheric Environment. 33(2): 159-173. [Korean Literature] https://doi.org/10.5572/KOSAE.2017.33.2.159
  19. Kryza M, Werner M, Dore AJ, Vieno M, Blas M, Drzeniecka OA, Netzel P. 2012. Modelling meteorological conditions for the episode (December 2009) of measured high $PM_{10}$ air concentrations in SW Poland - application of the WRF model, International Journal of Environment and Pollution. 50: 41-52. https://doi.org/10.1504/IJEP.2012.051179
  20. McNally DE. 2009. 12km MM5 Performance Goals. Presentation to the Ad-Hoc Meteorology Group. 25-June.
  21. Miglietta MM, Thunix P, Georgieva E, Pederzoli A, Bessagnet B, Terrenoire E, Colette A. 2012. Evaluation of WRF model performance in different European regions with the DELTAFAIRMODE evaluation tool. International Journal of Environment and Pollution. 50: 83-97. https://doi.org/10.1504/IJEP.2012.051183
  22. National Institute of Environmental Research (NIER). 2010. A study of data accuracy improvement for national air quality forecasting(III). [Korean Literature]
  23. National Institute of Environmental Research (NIER). 2013. Studies on the optimization method for improving the accuracy of air quality modeling. [Korean Literature]
  24. National Institute of Environmental Research (NIER). 2016. A study of accuracy improvement of numerical air quality forecasting model(III). [Korean Literature]
  25. National Institute of Environmental Research (NIER). 2017. A study of accuracy improvement of numerical air quality forecasting model(III). [Korean Literature]
  26. National Research Council (NRC). 2007. Models in Environmental Regulatory Decision Making. Washington, DC: National Research Council of the National Academies. doi:10.17226/11972.
  27. Scire JS, Robe FR, Fernau ME, Yamartino RJ. 2000. A User's Guide for the CALMET Meteorological Model(Version 5). Earth Tech, Inc.
  28. Skamarock WC, Klemp JB, Dudhia J, Gill DO, Barker DM, Duda MG, Huang X, Wang W, Powers JG. 2008. A description of the advanced research WRF version 3 (Note NCAR/TN-475+STR). National Center For Atmospheric Research Boulder Co Mesoscale and Micro-scale Meteorology Division.
  29. U.S. EPA (Environmental Protection Agency). 2004. The Ozone Report: Measuring Progress Through 2003. EPA 454/k-04-001. http://www.epa.gov/air/airtrends/aqtrnd04/ozone.html.
  30. U.S. EPA (Environmental Protection Agency). 2005. Technical Support Document for the Final Clean Air Interstate Rule-Air Quality Modeling. March 2005. Docket number OAR-2003-0053-0162. http://www.epa.gov/CAIR/pdfs/finaltech02.pdf.
  31. U.S. EPA (Environmental Protection Agency). 2006. Office of Air Quality Planning and Standards, Technical Support Document for the Proposed PM NAAQS Rule Response Surface Modeling. Research Triangle Park. NC 27711, February 2006.
  32. U.S. EPA (Environmental Protection Agency). 2007. Guidance on the use of models and other analyses for demonstrating attainment of air quality goals for ozone, $PM_{2.5}$, and regional haze. Tech Rep. EPA-454/B-07-002. Research Triangle Park. NC.
  33. Yarwood G, Morris RE, Wilson GM. 2007. Particulate matter source apportionment technology (PSAT) in the CAMx photochemical grid model. Air Pollution Modeling and its Application XVII. 478-492.