Journal of Korean Society of Coastal and Ocean Engineers (한국해안·해양공학회논문집)

Korean Society of Coastal and Ocean Engineers (한국해안·해양공학회)
권호 표지
DOI QR코드

DOI QR Code

Long-gap Filling Method for the Coastal Monitoring Data

해양모니터링 자료의 장기결측 보충 기법

  • Cho, Hong-Yeon (Marine Big-data Center, Korea Institute of Ocean Science and Technology, University of Science and Technology(UST)) ;
  • Lee, Gi-Seop (Marine Big-data Center, Korea Institute of Ocean Science and Technology) ;
  • Lee, Uk-Jae (Marine Big-data Center, Korea Institute of Ocean Science and Technology)
  • 조홍연 (한국해양과학기술원 해양빅데이터센터) ;
  • 이기섭 (한국해양과학기술원 해양빅데이터센터) ;
  • 이욱재 (한국해양과학기술원 해양빅데이터센터)
  • Received : 2021.11.15
  • Accepted : 2021.12.20
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Technique for the long-gap filling that occur frequently in ocean monitoring data is developed. The method estimates the unknown values of the long-gap by the summation of the estimated trend and selected residual components of the given missing intervals. The method was used to impute the data of the long-term missing interval of about 1 month, such as temperature and water temperature of the Ulleungdo ocean buoy data. The imputed data showed differences depending on the monitoring parameters, but it was found that the variation pattern was appropriately reproduced. Although this method causes bias and variance errors due to trend and residual components estimation, it was found that the bias error of statistical measure estimation due to long-term missing is greatly reduced. The mean, and the 90% confidence intervals of the gap-filling model's RMS errors are 0.93 and 0.35~1.95, respectively.

해양모니터링 자료에서 빈번하게 발생하는 장기결측구간의 자료 보충기법을 제안한다. 제안하는 방법은 결측구간의 장기변동 추세 성분과 단기변동 잔차성분을 추정하여 조합하는 방식으로 결측구간의 미지 정보를 추정한다. 이 방법을 이용하여 울릉도 해상부이 자료의 수온 항목, 약 1개월 정도의 장기결측 구간의 자료를 보충하였으며, 부이에서 관측하는 자료 항목에 대해서도 결측 보충을 수행하였다. 보충된 자료는 항목에 따라 차이를 보이지만 변동양상이 적절하게 재현되는 것으로 파악되었다. 이 방법은 추세추정과 잔차 반영에 따른 편향오차와 분산오차가 발생하지만, 장기결측으로 인한 통계적인 측도 추정의 편향오차는 크게 절감하는 것으로 파악되었다. 결측보충 모형의 추정 RMS 오차의 평균과 90% 신뢰구간은 각각 0.93, 0.35~1.95 범위이다.

Keywords

Acknowledgement

본 연구는 독도연구사업(PG-52262)의 지원을 받아 수행되었으며, 연구비 지원에 감사드립니다. 또한 해상 모니터링 자료를 제공해주신 기상청에 감사드립니다.

References

  1. Afrifa-Yamoah, E., Mueller, U.A., Taylor, S.M. and Fisher, A.J. (2020). Missing data imputation of high-resolution temporal climate time series data. Meteorological Applications, https://doi.org/10.1002/met.1873.
  2. Almendra-martin, L., Martinez-Fernandez, J., Piles, M. and Gonzalez-Zamora, A. (2021). Comparison of gap-filling techniques applied to the CCI soil moisture database in Southern Europe. Remote Sensing of Environment, 258, https://doi.org/10.1016/j.rse.2021.112377.
  3. Baddoo, T.D., Li, Z., Odai, S.N., Boni. K.R.C., Nooni, I.K. and Andam-Akorful, S.A. (2021). Comparison of missing data infilling mechanisms for recovering a real-world single station streamflow observation. International J. of Environmental research and Public Health, 18, https://doi.org/10.3390/ijerph18168375.
  4. Bellido-Jimenez, J.A., Gualda, J.E. and Garcia-Marin, A.P. (2021). Assessing machine learning models for gap-filling daily rainfall series in a semiarid region of Spain, Atmosphere, 12, 1158. https://doi.org/10.3390/atmos12091158.
  5. Cho, H.Y., Oh, J.H., Kom, K.O. and Shin, J.S. (2013). Outlier detection and missing data filling methods for coastal water temperature data. J. of Coastal Research, Sp[ecial Issue No. 65, 1898-1903.
  6. Grolemund, G. and Wickham, H. (2011). Dates and Times Made Easy with lubridate. Journal of Statistical Software, 40(3), 1-25. https://www.jstatsoft.org/v40/i03/.
  7. Hair, Jr. J.F., Black, W.C., Babin, B.J. and Anderson, R.E. (2010). Multivariate Data Analysis, A Global Perspective, Seventh Edition, Chapter 2, Pearson.
  8. Kandasamy, S., Baret, F., Verger, A., Neveux, P. and Weiss, M. (2013). A comparison of methods for smoothing and gap filling time series of remote sensing observations - application to MODIS LAI products. Biogeosciences, 10, 4055-4071, https://doi.org/10.5194/bg-10-4055-2013.
  9. Kang, M., Ichii, K., Kim, J., Indrawati, Y.M., Park, J., Moon, M., Lim, J.-H. and Chun, J.-H. (2019). New gap-filling strategies for long-period flux data gaps using a data-driven approach. Atmosphere, 10, 568, https://doi.org/10.3390/atmos10100568.
  10. Liu, X. and Wang, M. (2019). Filling the gaps of missing data in the merged VIIRS SNPP/NOAA-20 ocean color product using DINEOF method. Remote Sensing, 11, https://doi.org/10.3390/rs11020178.
  11. Millard, S.P. (2013). EnvStats: An R Package for Environmental Statistics. Springer, New York.
  12. Moritz, S and Bartz-Beielstein, T. (2017). imputeTS: Time Series Missing Value Imputation in R. The R Journal, 9(1), 207-218. https://doi.org/10.32614/RJ-2017-009.
  13. Golyandina, N. and Korobeynikov, A. (2014) Basic Singular Spectrum Analysis and Forecasting with R. Computational Statistics and Data Analysis, 71, 934-954. https://doi.org/10.1016/j.csda.2013.04.009
  14. Fredj, E., Roarty, H., Kohut, J., Smith, M. and Glenn, S. (2016). Gap filling of the coastal ocean surface currents from HFR data: Application to the Mid-Atlantic Bight HFR Network. Journal of Atmospheric and Oceanic Technology, 33(6), 1097-1111. https://doi.org/10.1175/JTECH-D-15-0056.1
  15. Rumaling, M.I., Chee, F.P., Dayou, J., Chang, J.H.W., Kong, S.S.K. and Sentian, J. (2020). Missing value imputation for PM10 concentration in Sabah using nearest neighbour method (NNM) and expectation-maximization (EM) algorithm. Asian Journal of Atmospheric Environment, 14(1), 62-72. https://doi.org/10.5572/ajae.2020.14.1.062
  16. Sarafanov, M., Kazakov, E., Nikitin, N.O. and Kalyuzhnaya, A.V. (2020). A machine learning approach for remote sensing data gap-filling with open-source implementation: An example regarding land surface temperature, surface albedo and NDVI. Remote Sensing, 12, https://doi.org/10.3390/rs12233865.
  17. Sattari, M.T., Falsafian, K., Irvem, A., Shahav, S. and Qasem, S.N. (2020) Potential of kernel and tree-based machine-learning models for estimating missing data of rainfall. Engineering Applications of Computational Fluid Mechanics, 14(1), 1078-1094. https://doi.org/10.1080/19942060.2020.1803971.
  18. Sim, J., Lee, J.S. and Kwon, B. (2015). Missing values and optimal selection of an imputation method and classfication algorothm to improve the accuracy of ubiquitous computing applications. Mathematical Problems in Engineering, 2015, http://dx.doi.org/10.1155/2015/538613.
  19. Velasco-Gallego, C. and Lazakis, I. (2020). Real-time data-driven missing data imputation for short-term sensor data of marine systems. A comparative study, Ocean Engineering, 218. https://doi.org/10.1016/j.oceaneng.2020.108261.
  20. Wand, M.P. (2021). KernSmooth: Functions for Kernel Smoothing Supporting Wand & Jones (1995). R package version 2.23-20. https://CRAN.R-project.org/package=KernSmooth.
  21. Wand, M.P. and Jones, M.C. (1995). Kernel Smoothing. Chapman and Hall, London.
  22. Wang, G., Ma, M., Jinag, L., Chen, F. and Xu, L. (2021). Multiple imputation of marine search and rescue data at multiple missingpatterns. PLOS ONE, 16(6), https://doi.org/10.1371/journal.pone.0252129.
  23. Zhao, X. and Huang, Y. (2015). A comparison of the three gap filling techniques for eddy covariance net carbon fluxes in short vegetation ecosystems, 2015. http://dx.doi.org/10.1155/2015/260580.