• 제목/요약/키워드: Anomaly Prediction

Search Result 107, Processing Time 0.023 seconds

Design and evaluation of artificial intelligence models for abnormal data detection and prediction

  • Hae-Jong Joo;Ho-Bin Song
    • Journal of Platform Technology
    • /
    • v.11 no.6
    • /
    • pp.3-12
    • /
    • 2023
  • In today's system operation, it is difficult to detect failures and take immediate action in the case of a shortage of manpower compared to the number of equipment or failures in vulnerable time zones, which can lead to delays in failure recovery. In addition, various algorithms exist to detect abnormal symptom data, and it is important to select an appropriate algorithm for each problem. In this paper, an ensemble-based isolation forest model was used to efficiently detect multivariate point anomalies that deviated from the mean distribution in the data set generated to predict system failure and minimize service interruption. And since significant changes in memory space usage are observed together with changes in CPU usage, the problem is solved by using LSTM-Auto Encoder for a collective anomaly in which another feature exhibits an abnormal pattern according to a change in one by comparing two or more features. did In addition, evaluation indicators are set for the performance evaluation of the model presented in this study, and then AI model evaluation is performed.

  • PDF

AN ANOMALY DETECTION METHOD BY ASSOCIATIVE CLASSIFICATION

  • Lee, Bum-Ju;Lee, Heon-Gyu;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.301-304
    • /
    • 2005
  • For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques or frequent episode mining in order to analyze an audit data. But, since they mainly analyze the average behaviour of user's activities, some anomalies can be detected inaccurately. Therefore, we propose an anomaly detection method that utilizes an associative classification for modelling intrusion detection. Finally, we proof that a prediction model built from associative classification method yields better accuracy than a prediction model built from a traditional methods by experimental results.

  • PDF

Electrical resistivity tomography survey for prediction of anomaly in mechanized tunneling

  • Lee, Kang-Hyun;Park, Jin-Ho;Park, Jeongjun;Lee, In-Mo;Lee, Seok-Won
    • Geomechanics and Engineering
    • /
    • v.19 no.1
    • /
    • pp.93-104
    • /
    • 2019
  • Anomalies and/or fractured grounds not detected by the surface geophysical and geological survey performed during design stage may cause significant problems during tunnel excavation. Many studies on prediction methods of the ground condition ahead of the tunnel face have been conducted and applied in tunneling construction sites, such as tunnel seismic profiling and probe drilling. However, most such applications have focused on the drill and blast tunneling method. Few studies have been conducted for mechanized tunneling because of the limitation in the available space to perform prediction tests. This study aims to predict the ground condition ahead of the tunnel face in TBM tunneling by using an electrical resistivity tomography survey. It compared the characteristics of each electrode array and performed an investigation on in-situ tunnel boring machine TBM construction site environments. Numerical simulations for each electrode array were performed, to determine the proper electrode array to predict anomalies ahead of the tunnel face. The results showed that the modified dipole-dipole array is, compared to other arrays, the best for predicting the location and condition of an anomaly. As the borehole becomes longer, the measured data increase accordingly. Therefore, longer boreholes allow a more accurate prediction of the location and status of anomalies and complex grounds.

An Ensemble Model for Machine Failure Prediction (앙상블 모델 기반의 기계 고장 예측 방법)

  • Cheon, Kang Min;Yang, Jaekyung
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.43 no.1
    • /
    • pp.123-131
    • /
    • 2020
  • There have been a lot of studies in the past for the method of predicting the failure of a machine, and recently, a lot of researches and applications have been generated to diagnose the physical condition of the machine and the parts and to calculate the remaining life through various methods. Survival models are also used to predict plant failures based on past anomaly cycles. In particular, special machine that reflect the fluid flow and process characteristics of chemical plants are connected to hundreds or thousands of sensors, so there are not many factors that need to be considered, such as process and material data as well as application of derivative variables. In this paper, the data were preprocessed through time series anomaly detection based on unsupervised learning to predict the abnormalities of these special machine. Next, clustering results reflecting clustering-based data characteristics were applied to produce additional variables, and a learning data set was created based on the history of past facility abnormalities. Finally, the prediction methodology based on the supervised learning algorithm was applied, and the model update was confirmed to improve the accuracy of the prediction of facility failure. Through this, it is expected to improve the efficiency of facility operation by flexibly replacing the maintenance time and parts supply and demand by predicting abnormalities of machine and extracting key factors.

Impact of Snow Depth Initialization on Seasonal Prediction of Surface Air Temperature over East Asia for Winter Season (겨울철 동아시아 지역 기온의 계절 예측에 눈깊이 초기화가 미치는 영향)

  • Woo, Sung-Ho;Jeong, Jee-Hoon;Kim, Baek-Min;Kim, Seong-Joong
    • Atmosphere
    • /
    • v.22 no.1
    • /
    • pp.117-128
    • /
    • 2012
  • Does snow depth initialization have a quantitative impact on sub-seasonal to seasonal prediction skill? To answer this question, a snow depth initialization technique for seasonal forecast system has been implemented and the impact of the initialization on the seasonal forecast of surface air temperature during the wintertime is examined. Since the snow depth observation can not be directly used in the model simulation due to the large systematic bias and much smaller model variability, an anomaly rescaling method to the snow depth initialization is applied. Snow depth in the model is initialized by adding a rescaled snow depth observation anomaly to the model snow depth climatology. A suite of seasonal forecast is performed for each year in recent 12 years (1999-2010) with and without the snow depth initialization to evaluate the performance of the developed technique. The results show that the seasonal forecast of surface air temperature over East Asian region sensitively depends on the initial snow depth anomaly over the region. However, the sensitivity shows large differences for different timing of the initialization and forecast lead time. Especially, the snow depth anomaly initialized in the late winter (Mar. 1) is the most effective in modulating the surface air temperature anomaly after one month. The real predictability gained by the snow depth initialization is also examined from the comparison with observation. The gain of the real predictability is generally small except for the forecasting experiment in the early winter (Nov. 1), which shows some skillful forecasts. Implications of these results and future directions for further development are discussed.

A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection (TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구)

  • Lee, Seung Hoon;Kim, Yong Soo
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.3
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

LSTM-based Anomaly Detection on Big Data for Smart Factory Monitoring (스마트 팩토리 모니터링을 위한 빅 데이터의 LSTM 기반 이상 탐지)

  • Nguyen, Van Quan;Van Ma, Linh;Kim, Jinsul
    • Journal of Digital Contents Society
    • /
    • v.19 no.4
    • /
    • pp.789-799
    • /
    • 2018
  • This article presents machine learning based approach on Big data to analyzing time series data for anomaly detection in such industrial complex system. Long Short-Term Memory (LSTM) network have been demonstrated to be improved version of RNN and have become a useful aid for many tasks. This LSTM based model learn the higher level temporal features as well as temporal pattern, then such predictor is used to prediction stage to estimate future data. The prediction error is the difference between predicted output made by predictor and actual in-coming values. An error-distribution estimation model is built using a Gaussian distribution to calculate the anomaly in the score of the observation. In this manner, we move from the concept of a single anomaly to the idea of the collective anomaly. This work can assist the monitoring and management of Smart Factory in minimizing failure and improving manufacturing quality.

Large-scale Atmospheric Patterns associated with the 2018 Heatwave Prediction in the Korea-Japan Region using GloSea6

  • Jinhee Kang;Semin Yun;Jieun Wie;Sang-Min Lee;Johan Lee;Baek-Jo Kim;Byung-Kwon Moon
    • Journal of the Korean earth science society
    • /
    • v.45 no.1
    • /
    • pp.37-47
    • /
    • 2024
  • In the summer of 2018, the Korea-Japan (KJ) region experienced an extremely severe and prolonged heatwave. This study examines the GloSea6 model's prediction performance for the 2018 KJ heatwave event and investigates how its prediction skill is related to large-scale circulation patterns identified by the k-means clustering method. Cluster 1 pattern is characterized by a KJ high-pressure anomaly, Cluster 2 pattern is distinguished by an Eastern European high-pressure anomaly, and Cluster 3 pattern is associated with a Pacific-Japan pattern-like anomaly. By analyzing the spatial correlation coefficients between these three identified circulation patterns and GloSea6 predictions, we assessed the contribution of each circulation pattern to the heatwave lifecycle. Our results show that the Eastern European high-pressure pattern, in particular, plays a significant role in predicting the evolution of the development and peak phases of the 2018 KJ heatwave approximately two weeks in advance. Furthermore, this study suggests that an accurate representation of large-scale atmospheric circulations in upstream regions is a key factor in seasonal forecast models for improving the predictability of extreme weather events, such as the 2018 KJ heatwave.

A Robust Bayesian Probabilistic Matrix Factorization Model for Collaborative Filtering Recommender Systems Based on User Anomaly Rating Behavior Detection

  • Yu, Hongtao;Sun, Lijun;Zhang, Fuzhi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.9
    • /
    • pp.4684-4705
    • /
    • 2019
  • Collaborative filtering recommender systems are vulnerable to shilling attacks in which malicious users may inject biased profiles to promote or demote a particular item being recommended. To tackle this problem, many robust collaborative recommendation methods have been presented. Unfortunately, the robustness of most methods is improved at the expense of prediction accuracy. In this paper, we construct a robust Bayesian probabilistic matrix factorization model for collaborative filtering recommender systems by incorporating the detection of user anomaly rating behaviors. We first detect the anomaly rating behaviors of users by the modified K-means algorithm and target item identification method to generate an indicator matrix of attack users. Then we incorporate the indicator matrix of attack users to construct a robust Bayesian probabilistic matrix factorization model and based on which a robust collaborative recommendation algorithm is devised. The experimental results on the MovieLens and Netflix datasets show that our model can significantly improve the robustness and recommendation accuracy compared with three baseline methods.

ANOMALY DETECTION FOR AN ORAL HEALTH CARE APPLICATION USING ONE CLASS YOLOV3

  • JAEHUN, BAEK;SEUNGWON, KIM;DONGWOOK, SHIN
    • Journal of the Korean Society for Industrial and Applied Mathematics
    • /
    • v.26 no.4
    • /
    • pp.310-322
    • /
    • 2022
  • In this report, we apply an anomaly detection algorithm to a mobile oral health care application. In particular, we have investigated one class YOLOv3 as an anomaly detection model to classify pictures of mouths which will be used as inputs in the following machine learning model. We have achieved outstanding performances by proposing appropriate annotation strategies for our data sets and modifying the loss function. Moreover, the model can classify not only oral and non-oral pictures but also output preprocessed pictures that only contain the area around the lips by using the predicted bounding box. Thus, the model performs prediction and preprocessing simultaneously.