• Title/Summary/Keyword: Time-series Anomaly Detection

Search Result 58, Processing Time 0.024 seconds

Intrusion Detection Method Using Unsupervised Learning-Based Embedding and Autoencoder (비지도 학습 기반의 임베딩과 오토인코더를 사용한 침입 탐지 방법)

  • Junwoo Lee;Kangseok Kim
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.8
    • /
    • pp.355-364
    • /
    • 2023
  • As advanced cyber threats continue to increase in recent years, it is difficult to detect new types of cyber attacks with existing pattern or signature-based intrusion detection method. Therefore, research on anomaly detection methods using data learning-based artificial intelligence technology is increasing. In addition, supervised learning-based anomaly detection methods are difficult to use in real environments because they require sufficient labeled data for learning. Research on an unsupervised learning-based method that learns from normal data and detects an anomaly by finding a pattern in the data itself has been actively conducted. Therefore, this study aims to extract a latent vector that preserves useful sequence information from sequence log data and develop an anomaly detection learning model using the extracted latent vector. Word2Vec was used to create a dense vector representation corresponding to the characteristics of each sequence, and an unsupervised autoencoder was developed to extract latent vectors from sequence data expressed as dense vectors. The developed autoencoder model is a recurrent neural network GRU (Gated Recurrent Unit) based denoising autoencoder suitable for sequence data, a one-dimensional convolutional neural network-based autoencoder to solve the limited short-term memory problem that GRU can have, and an autoencoder combining GRU and one-dimensional convolution was used. The data used in the experiment is time-series-based NGIDS (Next Generation IDS Dataset) data, and as a result of the experiment, an autoencoder that combines GRU and one-dimensional convolution is better than a model using a GRU-based autoencoder or a one-dimensional convolution-based autoencoder. It was efficient in terms of learning time for extracting useful latent patterns from training data, and showed stable performance with smaller fluctuations in anomaly detection performance.

Design of Multi-Level Abnormal Detection System Suitable for Time-Series Data (시계열 데이터에 적합한 다단계 비정상 탐지 시스템 설계)

  • Chae, Moon-Chang;Lim, Hyeok;Kang, Namhi
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.6
    • /
    • pp.1-7
    • /
    • 2016
  • As new information and communication technologies evolve, security threats are also becoming increasingly intelligent and advanced. In this paper, we analyze the time series data continuously entered through a series of periods from the network device or lightweight IoT (Internet of Things) devices by using the statistical technique and propose a system to detect abnormal behaviors of the device or abnormality based on the analysis results. The proposed system performs the first level abnormal detection by using previously entered data set, thereafter performs the second level anomaly detection according to the trust bound configured by using stored time series data based on time attribute or group attribute. Multi-level analysis is able to improve reliability and to reduce false positives as well through a variety of decision data set.

Evaluation of Edge-Based Data Collection System through Time Series Data Optimization Techniques and Universal Benchmark Development (수집 데이터 기반 경량 이상 데이터 감지 알림 시스템 개발)

  • Woojin Cho;Jae-hoi Gu
    • The Journal of the Convergence on Culture Technology
    • /
    • v.10 no.1
    • /
    • pp.453-458
    • /
    • 2024
  • Due to global issues such as climate crisis and rising energy costs, there is an increasing focus on energy conservation and management. In the case of South Korea, approximately 53.5% of the total energy consumption comes from industrial complexes. In order to address this, we aimed to improve issues through the 'Shared Network Utility Plant' among companies using similar energy utilities to find energy-saving points. For effective energy conservation, various techniques are utilized, and stable data supply is crucial for the reliable operation of factories. Many anomaly detection and alert systems for checking the stability of data supply were dependent on Energy Management Systems (EMS), which had limitations. The construction of an EMS involves large-scale systems, making it difficult to implement in small factories with spatial and energy constraints. In this paper, we aim to overcome these challenges by constructing a data collection system and anomaly detection alert system on embedded devices that consume minimal space and power. We explore the possibilities of utilizing anomaly detection alert systems in typical institutions for data collection and study the construction process.

Efficient Anomaly Detection Through Confidence Interval Estimation Based on Time Series Analysis (시계열 분석 기반 신뢰구간 추정을 통한 효율적인 이상감지)

  • Kim, Yeong-Ju;Heo, You-Kyung;Park, Jin-Gwan;Jeong, Min-A
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.39C no.8
    • /
    • pp.708-715
    • /
    • 2014
  • In this paper, we suggest a method of realtime confidence interval estimation to detect abnormal states of sensor data. For realtime confidence interval estimation, the mean square errors of the exponential smoothing method and moving average method, two of the time series analysis method, where compared, and the moving average method with less errors was applied. When the sensor data passes the bounds of the confidence interval estimation, the administrator is notified through alarming. As the suggested method is for realtime anomaly detection in a ship, an Android terminal was adopted for better communication between the wireless sensor network and users. For safe navigation, an administrator can make decisions promptly and accurately upon emergency situation in a ship by referring to the anomaly detection information through realtime confidence interval estimation.

Anomaly detection in blade pitch systems of floating wind turbines using LSTM-Autoencoder (LSTM-Autoencoder를 이용한 부유식 풍력터빈 블레이드 피치 시스템의 이상징후 감지)

  • Seongpil Cho
    • Journal of Aerospace System Engineering
    • /
    • v.18 no.4
    • /
    • pp.43-52
    • /
    • 2024
  • This paper presents an anomaly detection system that uses an LSTM-Autoencoder model to identify early-stage anomalies in the blade pitch system of floating wind turbines. The sensor data used in power plant monitoring systems is primarily composed of multivariate time-series data for each component. Comprising two unidirectional LSTM networks, the system skillfully uncovers long-term dependencies hidden within sequential time-series data. The autoencoder mechanism, learning solely from normal state data, effectively classifies abnormal states. Thus, by integrating these two networks, the system can proficiently detect anomalies. To confirm the effectiveness of the proposed framework, a real multivariate time-series dataset collected from a wind turbine model was employed. The LSTM-autoencoder model showed robust performance, achieving high classification accuracy.

Analysis of detected anomalies in VOC reduction facilities using deep learning

  • Min-Ji Son;Myung Ho Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.4
    • /
    • pp.13-20
    • /
    • 2023
  • In this paper, the actual data of VOC reduction facilities was analyzed through a model that detects and predicts data anomalies. Using the USAD model, which shows stable performance in the field of anomaly detection, anomalies in real-time data are detected and sensors that cause anomalies are searched. In addition, we propose a method of predicting and warning, when abnormalities that time will occur by predicting future outliers with an auto-regressive model. The experiment was conducted with the actual data of the VOC reduction facility, and the anomaly detection test results showed high detection rates with precision, recall, and F1-score of 98.54%, 89.08%, and 93.57%, respectively. As a result, averaging of the precision, recall, and F1-score for 8 sensors of detection rates were 99.64%, 99.37%, and 99.63%. In addition, the Hamming loss obtained to confirm the validity of the detection experiment for each sensor was 0.0058, showing stable performance. And the abnormal prediction test result showed stable performance with an average absolute error of 0.0902.

Pattern Extraction of Manufacturing Time Series Data Using Matrix Profile (매트릭스 프로파일을 이용한 제조 시계열 데이터 패턴 추출)

  • Kim, Tae-hyun;Jin, Kyo-hong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.210-212
    • /
    • 2022
  • In the manufacturing industry, various sensors are attached to monitor the status of production facility. In many cases, the data obtained through these sensors is time series data. In order to determine whether the status of the production facility is abnormal, the process of extracting patterns from time series data must be preceded. Also various methods for extracting patterns from time series data are studied. In this paper, we use matrix profile algorithm to extract patterns from the collected multivariate time series data. Through this, the pattern of multi sensor data currently being collected from the CNC machine is extracted.

  • PDF

Experimental Study on Application of an Anomaly Detection Algorithm in Electric Current Datasets Generated from Marine Air Compressor with Time-series Features (시계열 특징을 갖는 선박용 공기 압축기 전류 데이터의 이상 탐지 알고리즘 적용 실험)

  • Lee, Jung-Hyung
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.27 no.1
    • /
    • pp.127-134
    • /
    • 2021
  • In this study, an anomaly detection (AD) algorithm was implemented to detect the failure of a marine air compressor. A lab-scale experiment was designed to produce fault datasets (time-series electric current measurements) for 10 failure modes of the air compressor. The results demonstrated that the temporal pattern of the datasets showed periodicity with a different period, depending on the failure mode. An AD model with a convolutional autoencoder was developed and trained based on a normal operation dataset. The reconstruction error was used as the threshold for AD. The reconstruction error was noted to be dependent on the AD model and hyperparameter tuning. The AD model was applied to the synthetic dataset, which comprised both normal and abnormal conditions of the air compressor for validation. The AD model exhibited good detection performance on anomalies showing periodicity but poor performance on anomalies resulting from subtle load changes in the motor.

Application of Symbolic Representation Method for Fault Detection and Clustering in Semiconductor Fabrication Processes (반도체공정 이상탐지 및 클러스터링을 위한 심볼릭 표현법의 적용)

  • Loh, Woong-Kee;Hong, Sang-Jeen
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.11
    • /
    • pp.806-818
    • /
    • 2009
  • Since the invention of the integrated circuit (IC) in 1950s, semiconductor technology has undergone dramatic development up to these days. A complete semiconductor is manufactured through a diversity of processes. For better semiconductor productivity, fault detection and classification (FDC) has been rigorously studied for finding faults even before the processes are completed. For FDC, various kinds of sensors are attached in many semiconductor manufacturing devices, and sensor values are collected in a periodic manner. The collection of sensor values consists of sequences of real numbers, and hence is regarded as a kind of time-series data. In this paper, we propose an algorithm for detecting and clustering faults in semiconductor processes. The proposed algorithm is a modification of the existing anomaly detection algorithm dealing with symbolically-represented time-series. The contributions of this paper are: (1) showing that a modification of the existing anomaly detection algorithm dealing with general time-series could be used for semiconductor process data and (2) presenting experimental results for improving correctness of fault detection and clustering. As a result of our experiment, the proposed algorithm caused neither false positive nor false negative.

A Predictive Bearing Anomaly Detection Model Using the SWT-SVD Preprocessing Algorithm (SWT-SVD 전처리 알고리즘을 적용한 예측적 베어링 이상탐지 모델)

  • So-hyang Bak;Kwanghoon Pio Kim
    • Journal of Internet Computing and Services
    • /
    • v.25 no.1
    • /
    • pp.109-121
    • /
    • 2024
  • In various manufacturing processes such as textiles and automobiles, when equipment breaks down or stops, the machines do not work, which leads to time and financial losses for the company. Therefore, it is important to detect equipment abnormalities in advance so that equipment failures can be predicted and repaired before they occur. Most equipment failures are caused by bearing failures, which are essential parts of equipment, and detection bearing anomaly is the essence of PHM(Prognostics and Health Management) research. In this paper, we propose a preprocessing algorithm called SWT-SVD, which analyzes vibration signals from bearings and apply it to an anomaly transformer, one of the time series anomaly detection model networks, to implement bearing anomaly detection model. Vibration signals from the bearing manufacturing process contain noise due to the real-time generation of sensor values. To reduce noise in vibration signals, we use the Stationary Wavelet Transform to extract frequency components and perform preprocessing to extract meaningful features through the Singular Value Decomposition algorithm. For experimental validation of the proposed SWT-SVD preprocessing method in the bearing anomaly detection model, we utilize the PHM-2012-Challenge dataset provided by the IEEE PHM Conference. The experimental results demonstrate significant performance with an accuracy of 0.98 and an F1-Score of 0.97. Additionally, to substantiate performance improvement, we conduct a comparative analysis with previous studies, confirming that the proposed preprocessing method outperforms previous preprocessing methods in terms of performance.