• 제목/요약/키워드: prediction of anomaly

검색결과 108건 처리시간 0.023초

Design and evaluation of artificial intelligence models for abnormal data detection and prediction

  • Hae-Jong Joo;Ho-Bin Song
    • Journal of Platform Technology
    • /
    • 제11권6호
    • /
    • pp.3-12
    • /
    • 2023
  • In today's system operation, it is difficult to detect failures and take immediate action in the case of a shortage of manpower compared to the number of equipment or failures in vulnerable time zones, which can lead to delays in failure recovery. In addition, various algorithms exist to detect abnormal symptom data, and it is important to select an appropriate algorithm for each problem. In this paper, an ensemble-based isolation forest model was used to efficiently detect multivariate point anomalies that deviated from the mean distribution in the data set generated to predict system failure and minimize service interruption. And since significant changes in memory space usage are observed together with changes in CPU usage, the problem is solved by using LSTM-Auto Encoder for a collective anomaly in which another feature exhibits an abnormal pattern according to a change in one by comparing two or more features. did In addition, evaluation indicators are set for the performance evaluation of the model presented in this study, and then AI model evaluation is performed.

  • PDF

AN ANOMALY DETECTION METHOD BY ASSOCIATIVE CLASSIFICATION

  • Lee, Bum-Ju;Lee, Heon-Gyu;Ryu, Keun-Ho
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 2005년도 Proceedings of ISRS 2005
    • /
    • pp.301-304
    • /
    • 2005
  • For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques or frequent episode mining in order to analyze an audit data. But, since they mainly analyze the average behaviour of user's activities, some anomalies can be detected inaccurately. Therefore, we propose an anomaly detection method that utilizes an associative classification for modelling intrusion detection. Finally, we proof that a prediction model built from associative classification method yields better accuracy than a prediction model built from a traditional methods by experimental results.

  • PDF

Electrical resistivity tomography survey for prediction of anomaly in mechanized tunneling

  • Lee, Kang-Hyun;Park, Jin-Ho;Park, Jeongjun;Lee, In-Mo;Lee, Seok-Won
    • Geomechanics and Engineering
    • /
    • 제19권1호
    • /
    • pp.93-104
    • /
    • 2019
  • Anomalies and/or fractured grounds not detected by the surface geophysical and geological survey performed during design stage may cause significant problems during tunnel excavation. Many studies on prediction methods of the ground condition ahead of the tunnel face have been conducted and applied in tunneling construction sites, such as tunnel seismic profiling and probe drilling. However, most such applications have focused on the drill and blast tunneling method. Few studies have been conducted for mechanized tunneling because of the limitation in the available space to perform prediction tests. This study aims to predict the ground condition ahead of the tunnel face in TBM tunneling by using an electrical resistivity tomography survey. It compared the characteristics of each electrode array and performed an investigation on in-situ tunnel boring machine TBM construction site environments. Numerical simulations for each electrode array were performed, to determine the proper electrode array to predict anomalies ahead of the tunnel face. The results showed that the modified dipole-dipole array is, compared to other arrays, the best for predicting the location and condition of an anomaly. As the borehole becomes longer, the measured data increase accordingly. Therefore, longer boreholes allow a more accurate prediction of the location and status of anomalies and complex grounds.

앙상블 모델 기반의 기계 고장 예측 방법 (An Ensemble Model for Machine Failure Prediction)

  • 천강민;양재경
    • 산업경영시스템학회지
    • /
    • 제43권1호
    • /
    • pp.123-131
    • /
    • 2020
  • There have been a lot of studies in the past for the method of predicting the failure of a machine, and recently, a lot of researches and applications have been generated to diagnose the physical condition of the machine and the parts and to calculate the remaining life through various methods. Survival models are also used to predict plant failures based on past anomaly cycles. In particular, special machine that reflect the fluid flow and process characteristics of chemical plants are connected to hundreds or thousands of sensors, so there are not many factors that need to be considered, such as process and material data as well as application of derivative variables. In this paper, the data were preprocessed through time series anomaly detection based on unsupervised learning to predict the abnormalities of these special machine. Next, clustering results reflecting clustering-based data characteristics were applied to produce additional variables, and a learning data set was created based on the history of past facility abnormalities. Finally, the prediction methodology based on the supervised learning algorithm was applied, and the model update was confirmed to improve the accuracy of the prediction of facility failure. Through this, it is expected to improve the efficiency of facility operation by flexibly replacing the maintenance time and parts supply and demand by predicting abnormalities of machine and extracting key factors.

겨울철 동아시아 지역 기온의 계절 예측에 눈깊이 초기화가 미치는 영향 (Impact of Snow Depth Initialization on Seasonal Prediction of Surface Air Temperature over East Asia for Winter Season)

  • 우성호;정지훈;김백민;김성중
    • 대기
    • /
    • 제22권1호
    • /
    • pp.117-128
    • /
    • 2012
  • Does snow depth initialization have a quantitative impact on sub-seasonal to seasonal prediction skill? To answer this question, a snow depth initialization technique for seasonal forecast system has been implemented and the impact of the initialization on the seasonal forecast of surface air temperature during the wintertime is examined. Since the snow depth observation can not be directly used in the model simulation due to the large systematic bias and much smaller model variability, an anomaly rescaling method to the snow depth initialization is applied. Snow depth in the model is initialized by adding a rescaled snow depth observation anomaly to the model snow depth climatology. A suite of seasonal forecast is performed for each year in recent 12 years (1999-2010) with and without the snow depth initialization to evaluate the performance of the developed technique. The results show that the seasonal forecast of surface air temperature over East Asian region sensitively depends on the initial snow depth anomaly over the region. However, the sensitivity shows large differences for different timing of the initialization and forecast lead time. Especially, the snow depth anomaly initialized in the late winter (Mar. 1) is the most effective in modulating the surface air temperature anomaly after one month. The real predictability gained by the snow depth initialization is also examined from the comparison with observation. The gain of the real predictability is generally small except for the forecasting experiment in the early winter (Nov. 1), which shows some skillful forecasts. Implications of these results and future directions for further development are discussed.

AONet: Attention network with optional activation for unsupervised video anomaly detection

  • Akhrorjon Akhmadjon Ugli Rakhmonov;Barathi Subramanian;Bahar Amirian Varnousefaderani;Jeonghong Kim
    • ETRI Journal
    • /
    • 제46권5호
    • /
    • pp.890-903
    • /
    • 2024
  • Anomaly detection in video surveillance is crucial but challenging due to the rarity of irregular events and ambiguity of defining anomalies. We propose a method called AONet that utilizes a spatiotemporal module to extract spatiotemporal features efficiently, as well as a residual autoencoder equipped with an attention network for effective future frame prediction in video anomaly detection. AONet utilizes a novel activation function called OptAF that combines the strengths of the ReLU, leaky ReLU, and sigmoid functions. Furthermore, the proposed method employs a combination of robust loss functions to address various aspects of prediction errors and enhance training effectiveness. The performance of the proposed method is evaluated on three widely used benchmark datasets. The results indicate that the proposed method outperforms existing state-of-the-art methods and demonstrates comparable performance, achieving area under the curve values of 97.0%, 86.9%, and 73.8% on the UCSD Ped2, CUHK Avenue, and ShanghaiTech Campus datasets, respectively. Additionally, the high speed of the proposed method enables its application to real-time tasks.

TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구 (A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection)

  • 이승훈;김용수
    • 품질경영학회지
    • /
    • 제50권3호
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

스마트 팩토리 모니터링을 위한 빅 데이터의 LSTM 기반 이상 탐지 (LSTM-based Anomaly Detection on Big Data for Smart Factory Monitoring)

  • ;;김진술
    • 디지털콘텐츠학회 논문지
    • /
    • 제19권4호
    • /
    • pp.789-799
    • /
    • 2018
  • 이 논문에서는 이러한 산업 단지 시스템에서의 비정상적인 동작이 일어날 때, 시간 계열의 데이터를 분석하기 위하여 Big 데이터를 이용한 접근을 기반으로 하는 머신 러닝을 보여줍니다. Long Short-Term Memory (LSTM) 네트워크는 향상된 RNN버전으로서 입증되었으며 많은 작업에 유용한 도움이 되었습니다. 이 LSTM 기반 모델은 시간적 패턴뿐만 아니라 더 높은 레벨의 시간적 특징을 학습 한 다음, 미래의 데이터를 예측하기 위해 예측 단계에 사용됩니다. 예측 오차는 예측 인자에 의해 예측 된 결과와 실제 예상되는 값의 차이입니다. 오차 분포 추정 모델은 가우스 분포를 사용하여 관찰 스코어의 이상을 계산합니다. 이러한 방식으로, 우리는 하나의 비정상적 데이터의 개념에서 집단적인 비정상적 데이터 개념으로 바뀌어 갑니다. 이 작업은 실패를 최소화하고 제조품질을 향상시키는 Smart Factory의 모니터링 및 관리를 지원할 수 있습니다.

Large-scale Atmospheric Patterns associated with the 2018 Heatwave Prediction in the Korea-Japan Region using GloSea6

  • Jinhee Kang;Semin Yun;Jieun Wie;Sang-Min Lee;Johan Lee;Baek-Jo Kim;Byung-Kwon Moon
    • 한국지구과학회지
    • /
    • 제45권1호
    • /
    • pp.37-47
    • /
    • 2024
  • In the summer of 2018, the Korea-Japan (KJ) region experienced an extremely severe and prolonged heatwave. This study examines the GloSea6 model's prediction performance for the 2018 KJ heatwave event and investigates how its prediction skill is related to large-scale circulation patterns identified by the k-means clustering method. Cluster 1 pattern is characterized by a KJ high-pressure anomaly, Cluster 2 pattern is distinguished by an Eastern European high-pressure anomaly, and Cluster 3 pattern is associated with a Pacific-Japan pattern-like anomaly. By analyzing the spatial correlation coefficients between these three identified circulation patterns and GloSea6 predictions, we assessed the contribution of each circulation pattern to the heatwave lifecycle. Our results show that the Eastern European high-pressure pattern, in particular, plays a significant role in predicting the evolution of the development and peak phases of the 2018 KJ heatwave approximately two weeks in advance. Furthermore, this study suggests that an accurate representation of large-scale atmospheric circulations in upstream regions is a key factor in seasonal forecast models for improving the predictability of extreme weather events, such as the 2018 KJ heatwave.

A Robust Bayesian Probabilistic Matrix Factorization Model for Collaborative Filtering Recommender Systems Based on User Anomaly Rating Behavior Detection

  • Yu, Hongtao;Sun, Lijun;Zhang, Fuzhi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제13권9호
    • /
    • pp.4684-4705
    • /
    • 2019
  • Collaborative filtering recommender systems are vulnerable to shilling attacks in which malicious users may inject biased profiles to promote or demote a particular item being recommended. To tackle this problem, many robust collaborative recommendation methods have been presented. Unfortunately, the robustness of most methods is improved at the expense of prediction accuracy. In this paper, we construct a robust Bayesian probabilistic matrix factorization model for collaborative filtering recommender systems by incorporating the detection of user anomaly rating behaviors. We first detect the anomaly rating behaviors of users by the modified K-means algorithm and target item identification method to generate an indicator matrix of attack users. Then we incorporate the indicator matrix of attack users to construct a robust Bayesian probabilistic matrix factorization model and based on which a robust collaborative recommendation algorithm is devised. The experimental results on the MovieLens and Netflix datasets show that our model can significantly improve the robustness and recommendation accuracy compared with three baseline methods.