• Title/Summary/Keyword: Time-series Anomaly Detection

Search Result 58, Processing Time 0.03 seconds

A Study on the Air Pollution Monitoring Network Algorithm Using Deep Learning (심층신경망 모델을 이용한 대기오염망 자료확정 알고리즘 연구)

  • Lee, Seon-Woo;Yang, Ho-Jun;Lee, Mun-Hyung;Choi, Jung-Moo;Yun, Se-Hwan;Kwon, Jang-Woo;Park, Ji-Hoon;Jung, Dong-Hee;Shin, Hye-Jung
    • Journal of Convergence for Information Technology
    • /
    • v.11 no.11
    • /
    • pp.57-65
    • /
    • 2021
  • We propose a novel method to detect abnormal data of specific symptoms using deep learning in air pollution measurement system. Existing methods generally detect abnomal data by classifying data showing unusual patterns different from the existing time series data. However, these approaches have limitations in detecting specific symptoms. In this paper, we use DeepLab V3+ model mainly used for foreground segmentation of images, whose structure has been changed to handle one-dimensional data. Instead of images, the model receives time-series data from multiple sensors and can detect data showing specific symptoms. In addition, we improve model's performance by reducing the complexity of noisy form time series data by using 'piecewise aggregation approximation'. Through the experimental results, it can be confirmed that anomaly data detection can be performed successfully.

IMPROVING GLOBAL SUPPLY CHAIN RISK IDENTIFICATION USING RCF

  • MYUNGHYUN, JUNG;SEYEON, LEE;MINJUNG, GIM;HYUNGJO, KIM;JAEHO, LEE
    • Journal of the Korean Society for Industrial and Applied Mathematics
    • /
    • v.26 no.4
    • /
    • pp.280-295
    • /
    • 2022
  • This paper contains an introduction to industrial problems, solutions, and results conducted with the Korea Association of Machinery Industry. The client company commissioned the problem of upgrading the method of identifying global supply risky items. Accordingly, the factors affecting the supply and demand of imported items in the global supply chain were identified and the method of selecting risky items was studied and delivered. Through research and discussions with the client companies, it is confirmed that the most suitable factors for identifying global supply risky items are 'import size', 'import dependence', and 'trend abnormality'. The meaning of each indicator is introduced, and risky items are selected using export/import data until October 2022. Through this paper, it is expected that countries and companies will be able to identify global supply risky items in advance and prepare for risks in the new normal situation: the economic situation caused by infectious diseases such as the COVID-19 pandemic; and the export/import regulation due to geopolitical problems. The client company will include in his report, the method presented in this paper and the risky items selected by the method.

Development of an intelligent IIoT platform for stable data collection (안정적 데이터 수집을 위한 지능형 IIoT 플랫폼 개발)

  • Woojin Cho;Hyungah Lee;Dongju Kim;Jae-hoi Gu
    • The Journal of the Convergence on Culture Technology
    • /
    • v.10 no.4
    • /
    • pp.687-692
    • /
    • 2024
  • The energy crisis is emerging as a serious problem around the world. In the case of Korea, there is great interest in energy efficiency research related to industrial complexes, which use more than 53% of total energy and account for more than 45% of greenhouse gas emissions in Korea. One of the studies is a study on saving energy through sharing facilities between factories using the same utility in an industrial complex called a virtual energy network plant and through transactions between energy producing and demand factories. In such energy-saving research, data collection is very important because there are various uses for data, such as analysis and prediction. However, existing systems had several shortcomings in reliably collecting time series data. In this study, we propose an intelligent IIoT platform to improve it. The intelligent IIoT platform includes a preprocessing system to identify abnormal data and process it in a timely manner, classifies abnormal and missing data, and presents interpolation techniques to maintain stable time series data. Additionally, time series data collection is streamlined through database optimization. This paper contributes to increasing data usability in the industrial environment through stable data collection and rapid problem response, and contributes to reducing the burden of data collection and optimizing monitoring load by introducing a variety of chatbot notification systems.

Detection of Abnormal CAN Messages Using Periodicity and Time Series Analysis (CAN 메시지의 주기성과 시계열 분석을 활용한 비정상 탐지 방법)

  • Se-Rin Kim;Ji-Hyun Sung;Beom-Heon Youn;Harksu Cho
    • The Transactions of the Korea Information Processing Society
    • /
    • v.13 no.9
    • /
    • pp.395-403
    • /
    • 2024
  • Recently, with the advancement of technology, the automotive industry has seen an increase in network connectivity. CAN (Controller Area Network) bus technology enables fast and efficient data communication between various electronic devices and systems within a vehicle, providing a platform that integrates and manages a wide range of functions, from core systems to auxiliary features. However, this increased connectivity raises concerns about network security, as external attackers could potentially gain access to the automotive network, taking control of the vehicle or stealing personal information. This paper analyzed abnormal messages occurring in CAN and confirmed that message occurrence periodicity, frequency, and data changes are important factors in the detection of abnormal messages. Through DBC decoding, the specific meanings of CAN messages were interpreted. Based on this, a model for classifying abnormalities was proposed using the GRU model to analyze the periodicity and trend of message occurrences by measuring the difference (residual) between the predicted and actual messages occurring within a certain period as an abnormality metric. Additionally, for multi-class classification of attack techniques on abnormal messages, a Random Forest model was introduced as a multi-classifier using message occurrence frequency, periodicity, and residuals, achieving improved performance. This model achieved a high accuracy of over 99% in detecting abnormal messages and demonstrated superior performance compared to other existing models.

Estimation Method of Predicted Time Series Data Based on Absolute Maximum Value (최대 절대값 기반 시계열 데이터 예측 모델 평가 기법)

  • Shin, Ki-Hoon;Kim, Chul;Nam, Sang-Hun;Park, Sung-Jae;Yoo, Sung-Soo
    • Journal of Energy Engineering
    • /
    • v.27 no.4
    • /
    • pp.103-110
    • /
    • 2018
  • In this paper, we introduce evaluation method of time series prediction model with new approach of Mean Absolute Percentage Error(hereafter MAPE) and Symmetric Mean Absolute Percentage Error(hereafter sMAPE). There are some problems using MAPE and sMAPE. First MAPE can't evaluate Zero observation of dataset. Moreover, when the observed value is very close to zero it evaluate heavier than other methods. Finally it evaluate different measure even same error between observations and predicted values. And sMAPE does different evaluations are made depending on whether the same error value is over-predicted or under-predicted. And it has different measurement according to the each sign, even if error is the same distance. These problems were solved by Maximum Mean Absolute Percentage Error(hereafter mMAPE). we used the absolute maximum of observed value as denominator instead of the observed value in MAPE, when the value is less than 1, removed denominator then solved the problem that the zero value is not defined. and were able to prevent heavier measurement problem. Also, if the absolute maximum of observed value is greater than 1, the evaluation values of mMAPE were compared with those of the other evaluations. With Beijing PM2.5 temperature data and our simulation data, we compared the evaluation values of mMAPE with other evaluations. And we proved that mMAPE can solve the problems that we mentioned.

The Fault Diagnosis Model of Ship Fuel System Equipment Reflecting Time Dependency in Conv1D Algorithm Based on the Convolution Network (합성곱 네트워크 기반의 Conv1D 알고리즘에서 시간 종속성을 반영한 선박 연료계통 장비의 고장 진단 모델)

  • Kim, Hyung-Jin;Kim, Kwang-Sik;Hwang, Se-Yun;Lee, Jang Hyun
    • Journal of Navigation and Port Research
    • /
    • v.46 no.4
    • /
    • pp.367-374
    • /
    • 2022
  • The purpose of this study was to propose a deep learning algorithm that applies to the fault diagnosis of fuel pumps and purifiers of autonomous ships. A deep learning algorithm reflecting the time dependence of the measured signal was configured, and the failure pattern was trained using the vibration signal, measured in the equipment's regular operation and failure state. Considering the sequential time-dependence of deterioration implied in the vibration signal, this study adopts Conv1D with sliding window computation for fault detection. The time dependence was also reflected, by transferring the measured signal from two-dimensional to three-dimensional. Additionally, the optimal values of the hyper-parameters of the Conv1D model were determined, using the grid search technique. Finally, the results show that the proposed data preprocessing method as well as the Conv1D model, can reflect the sequential dependency between the fault and its effect on the measured signal, and appropriately perform anomaly as well as failure detection, of the equipment chosen for application.

Calibration and Validation Activities for Earth Observation Mission Future Evolution for GMES

  • LECOMTE Pascal
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.237-240
    • /
    • 2005
  • Calibration and Validation are major element of any space borne Earth Observation Mission. These activities are the major objective of the commissioning phases but routine activities shall be maintained during the whole mission in order to maintain the quality of the product delivered to the users or at least to fully characterise the evolution with time of the product quality. With the launch of ERS-l in 1991, the European Space Agency decided to put in place a group dedicated to these activities, along with the daily monitoring of the product quality for anomaly detection and algorithm evolution. These four elements are all strongly linked together. Today this group is fully responsible for the monitoring of two ESA missions, ERS-2 and Envisat, for a total of 12 instruments of various types, preparing itself for the Earth Explorer series of five. other satellites (Cryosat, Goce, SMOS, ADM-Aeolus, Swarm) and at various levels in past and future Third Party Missions such as Landsat, J-ERS, ALOS and KOMPSAT. The Joint proposal by the European Union and the European Space Agency for a 'Global Monitoring for Environment and Security' project (GMES), triggers a review of the scope of these activities in a much wider framework than the handling of single missions with specific tools, methods and activities. Because of the global objective of this proposal, it is necessary to put in place Multi-Mission Calibration and Validation systems and procedures. GMES Calibration and Validation activities will rely on multi source data access, interoperability, long-term data preservation, and definition standards to facilitate the above objectives. The scope of this presentation is to give an overview of the current Calibration and Validation activities at ESA, and the planned evolution in the context of GMES.

  • PDF

Development of a complex failure prediction system using Hierarchical Attention Network (Hierarchical Attention Network를 이용한 복합 장애 발생 예측 시스템 개발)

  • Park, Youngchan;An, Sangjun;Kim, Mintae;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.26 no.4
    • /
    • pp.127-148
    • /
    • 2020
  • The data center is a physical environment facility for accommodating computer systems and related components, and is an essential foundation technology for next-generation core industries such as big data, smart factories, wearables, and smart homes. In particular, with the growth of cloud computing, the proportional expansion of the data center infrastructure is inevitable. Monitoring the health of these data center facilities is a way to maintain and manage the system and prevent failure. If a failure occurs in some elements of the facility, it may affect not only the relevant equipment but also other connected equipment, and may cause enormous damage. In particular, IT facilities are irregular due to interdependence and it is difficult to know the cause. In the previous study predicting failure in data center, failure was predicted by looking at a single server as a single state without assuming that the devices were mixed. Therefore, in this study, data center failures were classified into failures occurring inside the server (Outage A) and failures occurring outside the server (Outage B), and focused on analyzing complex failures occurring within the server. Server external failures include power, cooling, user errors, etc. Since such failures can be prevented in the early stages of data center facility construction, various solutions are being developed. On the other hand, the cause of the failure occurring in the server is difficult to determine, and adequate prevention has not yet been achieved. In particular, this is the reason why server failures do not occur singularly, cause other server failures, or receive something that causes failures from other servers. In other words, while the existing studies assumed that it was a single server that did not affect the servers and analyzed the failure, in this study, the failure occurred on the assumption that it had an effect between servers. In order to define the complex failure situation in the data center, failure history data for each equipment existing in the data center was used. There are four major failures considered in this study: Network Node Down, Server Down, Windows Activation Services Down, and Database Management System Service Down. The failures that occur for each device are sorted in chronological order, and when a failure occurs in a specific equipment, if a failure occurs in a specific equipment within 5 minutes from the time of occurrence, it is defined that the failure occurs simultaneously. After configuring the sequence for the devices that have failed at the same time, 5 devices that frequently occur simultaneously within the configured sequence were selected, and the case where the selected devices failed at the same time was confirmed through visualization. Since the server resource information collected for failure analysis is in units of time series and has flow, we used Long Short-term Memory (LSTM), a deep learning algorithm that can predict the next state through the previous state. In addition, unlike a single server, the Hierarchical Attention Network deep learning model structure was used in consideration of the fact that the level of multiple failures for each server is different. This algorithm is a method of increasing the prediction accuracy by giving weight to the server as the impact on the failure increases. The study began with defining the type of failure and selecting the analysis target. In the first experiment, the same collected data was assumed as a single server state and a multiple server state, and compared and analyzed. The second experiment improved the prediction accuracy in the case of a complex server by optimizing each server threshold. In the first experiment, which assumed each of a single server and multiple servers, in the case of a single server, it was predicted that three of the five servers did not have a failure even though the actual failure occurred. However, assuming multiple servers, all five servers were predicted to have failed. As a result of the experiment, the hypothesis that there is an effect between servers is proven. As a result of this study, it was confirmed that the prediction performance was superior when the multiple servers were assumed than when the single server was assumed. In particular, applying the Hierarchical Attention Network algorithm, assuming that the effects of each server will be different, played a role in improving the analysis effect. In addition, by applying a different threshold for each server, the prediction accuracy could be improved. This study showed that failures that are difficult to determine the cause can be predicted through historical data, and a model that can predict failures occurring in servers in data centers is presented. It is expected that the occurrence of disability can be prevented in advance using the results of this study.