• Title/Summary/Keyword: Abnormal Data

Search Result 1,611, Processing Time 0.029 seconds

Approach to diagnosing multiple abnormal events with single-event training data

  • Ji Hyeon Shin;Seung Gyu Cho;Seo Ryong Koo;Seung Jun Lee
    • Nuclear Engineering and Technology
    • /
    • v.56 no.2
    • /
    • pp.558-567
    • /
    • 2024
  • Diagnostic support systems are being researched to assist operators in identifying and responding to abnormal events in a nuclear power plant. Most studies to date have considered single abnormal events only, for which it is relatively straightforward to obtain data to train the deep learning model of the diagnostic support system. However, cases in which multiple abnormal events occur must also be considered, for which obtaining training data becomes difficult due to the large number of combinations of possible abnormal events. This study proposes an approach to maintain diagnostic performance for multiple abnormal events by training a deep learning model with data on single abnormal events only. The proposed approach is applied to an existing algorithm that can perform feature selection and multi-label classification. We choose an extremely randomized trees classifier to select dedicated monitoring parameters for target abnormal events. In diagnosing each event occurrence independently, two-channel convolutional neural networks are employed as sub-models. The algorithm was tested in a case study with various scenarios, including single and multiple abnormal events. Results demonstrated that the proposed approach maintained diagnostic performance for 15 single abnormal events and significantly improved performance for 105 multiple abnormal events compared to the base model.

ADA: Advanced data analytics methods for abnormal frequent episodes in the baseline data of ISD

  • Biswajit Biswal;Andrew Duncan;Zaijing Sun
    • Nuclear Engineering and Technology
    • /
    • v.54 no.11
    • /
    • pp.3996-4004
    • /
    • 2022
  • The data collected by the In-Situ Decommissioning (ISD) sensors are time-specific, age-specific, and developmental stage-specific. Research has been done on the stream data collected by ISD testbed in the recent few years to seek both frequent episodes and abnormal frequent episodes. Frequent episodes in the data stream have confirmed the daily cycle of the sensor responses and established sequences of different types of sensors, which was verified by the experimental setup of the ISD Sensor Network Test Bed. However, the discovery of abnormal frequent episodes remained a challenge because these abnormal frequent episodes are very small signals and may be buried in the background noise of voltage and current changes. In this work, we proposed Advanced Data Analytics (ADA) methods that are applied to the baseline data to identify frequent episodes and extended our approach by adding more features extracted from the baseline data to discover abnormal frequent episodes, which may lead to the early indicators of ISD system failures. In the study, we have evaluated our approach using the baseline data, and the performance evaluation results show that our approach is able to discover frequent episodes as well as abnormal frequent episodes conveniently.

Calculated Damage of Italian Ryegrass in Abnormal Climate Based World Meteorological Organization Approach Using Machine Learning

  • Jae Seong Choi;Ji Yung Kim;Moonju Kim;Kyung Il Sung;Byong Wan Kim
    • Journal of The Korean Society of Grassland and Forage Science
    • /
    • v.43 no.3
    • /
    • pp.190-198
    • /
    • 2023
  • This study was conducted to calculate the damage of Italian ryegrass (IRG) by abnormal climate using machine learning and present the damage through the map. The IRG data collected 1,384. The climate data was collected from the Korea Meteorological Administration Meteorological data open portal.The machine learning model called xDeepFM was used to detect IRG damage. The damage was calculated using climate data from the Automated Synoptic Observing System (95 sites) by machine learning. The calculation of damage was the difference between the Dry matter yield (DMY)normal and DMYabnormal. The normal climate was set as the 40-year of climate data according to the year of IRG data (1986~2020). The level of abnormal climate was set as a multiple of the standard deviation applying the World Meteorological Organization (WMO) standard. The DMYnormal was ranged from 5,678 to 15,188 kg/ha. The damage of IRG differed according to region and level of abnormal climate with abnormal temperature, precipitation, and wind speed from -1,380 to 1,176, -3 to 2,465, and -830 to 962 kg/ha, respectively. The maximum damage was 1,176 kg/ha when the abnormal temperature was -2 level (+1.04℃), 2,465 kg/ha when the abnormal precipitation was all level and 962 kg/ha when the abnormal wind speed was -2 level (+1.60 ㎧). The damage calculated through the WMO method was presented as an map using QGIS. There was some blank area because there was no climate data. In order to calculate the damage of blank area, it would be possible to use the automatic weather system (AWS), which provides data from more sites than the automated synoptic observing system (ASOS).

Abnormal Data Augmentation Method Using Perturbation Based on Hypersphere for Semi-Supervised Anomaly Detection (준 지도 이상 탐지 기법의 성능 향상을 위한 섭동을 활용한 초구 기반 비정상 데이터 증강 기법)

  • Jung, Byeonggil;Kwon, Junhyung;Min, Dongjun;Lee, Sangkyun
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.4
    • /
    • pp.647-660
    • /
    • 2022
  • Recent works demonstrate that the semi-supervised anomaly detection method functions quite well in the environment with normal data and some anomalous data. However, abnormal data shortages can occur in an environment where it is difficult to reserve anomalous data, such as an unknown attack in the cyber security fields. In this paper, we propose ADA-PH(Abnormal Data Augmentation Method using Perturbation based on Hypersphere), a novel anomalous data augmentation method that is applicable in an environment where abnormal data is insufficient to secure the performance of the semi-supervised anomaly detection method. ADA-PH generates abnormal data by perturbing samples located relatively far from the center of the hypersphere. With the network intrusion detection datasets where abnormal data is rare, ADA-PH shows 23.63% higher AUC performance than anomaly detection without data augmentation and even performs better than the other augmentation methods. Also, we further conduct quantitative and qualitative analysis on whether generated abnormal data is anomalous.

Abnormal state diagnosis model tolerant to noise in plant data

  • Shin, Ji Hyeon;Kim, Jae Min;Lee, Seung Jun
    • Nuclear Engineering and Technology
    • /
    • v.53 no.4
    • /
    • pp.1181-1188
    • /
    • 2021
  • When abnormal events occur in a nuclear power plant, operators must conduct appropriate abnormal operating procedures. It is burdensome though for operators to choose the appropriate procedure considering the numerous main plant parameters and hundreds of alarms that should be judged in a short time. Recently, various research has applied deep-learning algorithms to support this problem by classifying each abnormal condition with high accuracy. Most of these models are trained with simulator data because of a lack of plant data for abnormal states, and as such, developed models may not have tolerance for plant data in actual situations. In this study, two approaches are investigated for a deep-learning model trained with simulator data to overcome the performance degradation caused by noise in actual plant data. First, a preprocessing method using several filters was employed to smooth the test data noise, and second, a data augmentation method was applied to increase the acceptability of the untrained data. Results of this study confirm that the combination of these two approaches can enable high model performance even in the presence of noisy data as in real plants.

An Abnormal Breakpoint Data Positioning Method of Wireless Sensor Network Based on Signal Reconstruction

  • Zhijie Liu
    • Journal of Information Processing Systems
    • /
    • v.19 no.3
    • /
    • pp.377-384
    • /
    • 2023
  • The existence of abnormal breakpoint data leads to poor channel balance in wireless sensor networks (WSN). To enhance the communication quality of WSNs, a method for positioning abnormal breakpoint data in WSNs on the basis of signal reconstruction is studied. The WSN signal is collected using compressed sensing theory; the common part of the associated data set is mined by exchanging common information among the cluster head nodes, and the independent parts are updated within each cluster head node. To solve the non-convergence problem in the distributed computing, the approximate term is introduced into the optimization objective function to make the sub-optimization problem strictly convex. And the decompressed sensing signal reconstruction problem is addressed by the alternating direction multiplier method to realize the distributed signal reconstruction of WSNs. Based on the reconstructed WSN signal, the abnormal breakpoint data is located according to the characteristic information of the cross-power spectrum. The proposed method can accurately acquire and reconstruct the signal, reduce the bit error rate during signal transmission, and enhance the communication quality of the experimental object.

Abnormal signal detection based on parallel autoencoders (병렬 오토인코더 기반의 비정상 신호 탐지)

  • Lee, Kibae;Lee, Chong Hyun
    • The Journal of the Acoustical Society of Korea
    • /
    • v.40 no.4
    • /
    • pp.337-346
    • /
    • 2021
  • Detection of abnormal signal generally can be done by using features of normal signals as main information because of data imbalance. This paper propose an efficient method for abnormal signal detection using parallel AutoEncoder (AE) which can use features of abnormal signals as well. The proposed Parallel AE (PAE) is composed of a normal and an abnormal reconstructors having identical AE structure and train features of normal and abnormal signals, respectively. The PAE can effectively solve the imbalanced data problem by sequentially training normal and abnormal data. For further detection performance improvement, additional binary classifier can be added to the PAE. Through experiments using public acoustic data, we obtain that the proposed PAE shows Area Under Curve (AUC) improvement of minimum 22 % at the expenses of training time increased by 1.31 ~ 1.61 times to the single AE. Furthermore, the PAE shows 93 % AUC improvement in detecting abnormal underwater acoustic signal when pre-trained PAE is transferred to train open underwater acoustic data.

A Visualization Scheme with a Calendar Heat Map for Abnormal Pattern Analysis in the Manufacturing Process

  • Chankhihort, Doung;Lim, Byung-Muk;Lee, Gyu-Jung;Choi, Sungsu;Kwon, Sun-Ock;Lee, Sang-Hyun;Kang, Jeong-Tae;Nasridinov, Aziz;Yoo, Kwan-Hee
    • International Journal of Contents
    • /
    • v.13 no.2
    • /
    • pp.21-28
    • /
    • 2017
  • Abnormal data in the manufacturing process makes it difficult to find useful information that can be applied in data management for the manufacturing industry. It causes various problems in the daily process of production. An issue from the abnormal data can be handled by our method that uses big data and visualization. Visualization is a new technology that transforms data representation into a two-dimensional representation. Nowadays, many newly developed technologies provide data analysis, algorithm, optimization, and high efficiency, and they meet user requirements. We propose combined production of the data visualization approach that uses integrative visualization of sources of abnormal pattern analysis results. The perceived idea of the proposed approach can solve the problem as it also works for big data. It can also improve the performance and understanding by using visualization and solving issues that occur in the manufacturing process with a calendar heat map.

Classification Abnormal temperatures based on Meteorological Environment using Random forests (랜덤포레스트를 이용한 기상 환경에 따른 이상기온 분류)

  • Youn Su Kim;Kwang Yoon Song;In Hong Chang
    • Journal of Integrative Natural Science
    • /
    • v.17 no.1
    • /
    • pp.1-12
    • /
    • 2024
  • Many abnormal climate events are occurring around the world. The cause of abnormal climate is related to temperature. Factors that affect temperature include excessive emissions of carbon and greenhouse gases from a global perspective, and air circulation from a local perspective. Due to the air circulation, many abnormal climate phenomena such as abnormally high temperature and abnormally low temperature are occurring in certain areas, which can cause very serious human damage. Therefore, the problem of abnormal temperature should not be approached only as a case of climate change, but should be studied as a new category of climate crisis. In this study, we proposed a model for the classification of abnormal temperature using random forests based on various meteorological data such as longitudinal observations, yellow dust, ultraviolet radiation from 2018 to 2022 for each region in Korea. Here, the meteorological data had an imbalance problem, so the imbalance problem was solved by oversampling. As a result, we found that the variables affecting abnormal temperature are different in different regions. In particular, the central and southern regions are influenced by high pressure (Mainland China, Siberian high pressure, and North Pacific high pressure) due to their regional characteristics, so pressure-related variables had a significant impact on the classification of abnormal temperature. This suggests that a regional approach can be taken to predict abnormal temperatures from the surrounding meteorological environment. In addition, in the event of an abnormal temperature, it seems that it is possible to take preventive measures in advance according to regional characteristics.

Pre-Evaluation for Detecting Abnormal Users in Recommender System

  • Lee, Seok-Jun;Kim, Sun-Ok;Lee, Hee-Choon
    • Journal of the Korean Data and Information Science Society
    • /
    • v.18 no.3
    • /
    • pp.619-628
    • /
    • 2007
  • This study is devoted to suggesting the norm of detection abnormal users who are inferior to the other users in the recommender system compared with estimation accuracy. To select the abnormal users, we propose the pre-filtering method by using the preference ratings to the item rated by users. In this study, the experimental result shows the possibility of detecting the abnormal users before the process of preference estimation through the prediction algorithm. And It will be possible to improve the performance of the recommender system by using this detecting norm.

  • PDF