Machine Learning-based Quality Control and Error Correction Using Homogeneous Temporal Data Collected by IoT Sensors

IoT센서로 수집된 균질 시간 데이터를 이용한 기계학습 기반의 품질관리 및 데이터 보정

Kim, Hye-Jin;Lee, Hyeon Soo;Choi, Byung Jin;Kim, Yong-Hyuk

  • Received : 2019.01.28
  • Accepted : 2019.04.20
  • Published : 2019.04.28


In this paper, quality control (QC) is applied to each meteorological element of weather data collected from seven IoT sensors such as temperature. In addition, we propose a method for estimating the data regarded as error by means of machine learning. The collected meteorological data was linearly interpolated based on the basic QC results, and then machine learning-based QC was performed. Support vector regression, decision table, and multilayer perceptron were used as machine learning techniques. We confirmed that the mean absolute error (MAE) of the machine learning models through the basic QC is 21% lower than that of models without basic QC. In addition, when the support vector regression model was compared with other machine learning methods, it was found that the MAE is 24% lower than that of the multilayer neural network and 58% lower than that of the decision table on average.


Convergence;Machine Learning;Quality Control;Data Correction;Weather Data


  1. N.-Y. Kim, Y.-H. Kim, Y. Yoon, H.-H. Im, R. K. Y. Choi, and Y. H. Lee. (2015). Correcting air-pressure data collected by MEMS sensors in smartphones. Journal of Sensors, Article ID 245498.
  2. M.-K. Lee, S.-H. Moon, Y. Yoon , Y.-H. Kim, and B.-R. Moon. (2018). Detecting anomalies in meteorological data using support vector regression. Advances in Meteorology, Article ID 5439256.
  3. J.-H. Ha, Y.-H. Kim, H.-H. Im, N.-Y. Kim, S. Sim, and Y. Yoon. (2018). Error correction of meteorological data obtained with Mini-AWSs based on machine learning. Advances in Meteorology, Article ID 7210137.
  4. Y.-H. Kim, J.-H. Ha, Y. Yoon, N.-Y. Kim, H.-H. Im, S. Sim, and R. K. Y. Choi. (2016). Improved correction of atmospheric pressure data obtained by smartphones through machine learning. Computational Intelligence and Neuroscience, Article ID 9467878.
  5. M.-K. Lee, S.-H. Moon, Y.-H. Kim, and B.-R. Moon. (2014. October). Correcting abnormalities in meteorological data by machine learning. IEEE International Conference on Systems, Man, and Cybernetics. (pp.888-893). San Diego : IEEE
  6. G.-D. Kim & Y.-H. Kim. (2018). Correction of drifter data using recurrent neural networks. Journal of the Korea Convergence Society, 9(3), 15-21.
  7. A. J. Smola & B. Scholkopf. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3), 199-222.
  8. U. W. Pooch. (1974). Translation of decision tables. ACM Computing Surveys, 6(2), 125-151.
  9. F. Rosenblatt (1961). Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Washington DC : Spartan Books.
  10. J. A. Suykens & J. Vandewalle. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9(3), 293-300.
  11. N. R. Draper & H. Smith. (1998). Applied Regression Analysis, Thirds Edition.Wiley.
  12. M. Riedmiller & H. Braun. (1993). A direct adaptive method for faster backpropagation learning: the RPROP algorithm. IEEE International Conference on Neural Networks.. (pp.586-591).
  13. R. Kohavi. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. International Joint Conference on Artificial Intelligence Organization, 14(2), 1137-1145. San Francisco : Morgan Kaufmann.
  14. E. Frank, M. A. Hall, and I. H. Witten. (2016). Data Mining: Practical Machine Learning Tools and Techniques, Fourth Edition. Morgan Kaufmann.
  15. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. (2009). The WEKA data mining software: an update. Newsletter of SIGKDD Explorations, 11(1), 10-18.


Supported by : Ministry of Land, Infrastructure and Transport