Journal of Platform Technology

ICT Platform Society (ICT플랫폼학회)
권호 표지

Design and evaluation of artificial intelligence models for abnormal data detection and prediction

  • Hae-Jong Joo (University of Kangnam, Dept. of KNU Cham-Injae College) ;
  • Ho-Bin Song (University of Mokwon, Dept. of Electrical & Electric Engineering)
  • Received : 2023.11.21
  • Accepted : 2023.12.13
  • Published : 2023.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

In today's system operation, it is difficult to detect failures and take immediate action in the case of a shortage of manpower compared to the number of equipment or failures in vulnerable time zones, which can lead to delays in failure recovery. In addition, various algorithms exist to detect abnormal symptom data, and it is important to select an appropriate algorithm for each problem. In this paper, an ensemble-based isolation forest model was used to efficiently detect multivariate point anomalies that deviated from the mean distribution in the data set generated to predict system failure and minimize service interruption. And since significant changes in memory space usage are observed together with changes in CPU usage, the problem is solved by using LSTM-Auto Encoder for a collective anomaly in which another feature exhibits an abnormal pattern according to a change in one by comparing two or more features. did In addition, evaluation indicators are set for the performance evaluation of the model presented in this study, and then AI model evaluation is performed.

Keywords

References

  1. M. S. Islam, W. Pourmajidi, L. Zhang, J. Steinbacher, T. Erwin and A. Miranskyy, "Anomaly Detection in a Large-Scale Cloud Platform", 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), Madrid, ES, 2021
  2. Tanja Hagemann, Katerina Katsarou, "Reconstruction-based anomaly detection for the cloud", 2020 4th International Conference on Cloud and Big Data Computing, 2020
  3. S. N. Shirazi, S. Simpson, A. Gouglidis, A. Mauthe and D. Hutchison, "Anomaly Detection in the Cloud Using Data Density", 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), San Francisco, CA, USA, 2016
  4. D. Smith, Q. Guan and S. Fu, "An Anomaly Detection Framework for Autonomic Management of Compute Cloud Systems", 2010 IEEE 34th Annual Computer Software and Applications Conference Workshops, Seoul, Korea (South), 2010
  5. X. Jin and X. Qiu, "An Adaptive Anomaly Detection Method for Cloud Computing System", 2022 IEEE 5th International Conference on Electronics Technology (ICET), Chengdu, China, 2022
  6. M. A. Bashar and R. Nayak, "TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks", 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, ACT, Australia, 2020
  7. A. Geiger, D. Liu, S. Alnegheimish, A. Cuesta-Infante and K. Veeramachaneni, "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks", 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020
  8. A. Lavin and S. Ahmad, "Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark", 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, 2015
  9. Kim, S., Choi, K., Choi, H.-S., Lee, B., & Yoon, S., "Towards a Rigorous Evaluation of Time-Series Anomaly Detection", AI Conference on Artificial Intelligence, 2022
  10. Deepthi Cheboli, Anomaly Detection of Time Series, IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master Of Science, 2010
  11. Subutai Ahmad, Alexander Lavin, Scott Purdy, Zuha Agha, "Unsupervised real-time anomaly detection for streaming data", Neurocomputing, Volume 262, 2017