• 제목/요약/키워드: Abnormal Data

검색결과 1,611건 처리시간 0.033초

Approach to diagnosing multiple abnormal events with single-event training data

  • Ji Hyeon Shin;Seung Gyu Cho;Seo Ryong Koo;Seung Jun Lee
    • Nuclear Engineering and Technology
    • /
    • 제56권2호
    • /
    • pp.558-567
    • /
    • 2024
  • Diagnostic support systems are being researched to assist operators in identifying and responding to abnormal events in a nuclear power plant. Most studies to date have considered single abnormal events only, for which it is relatively straightforward to obtain data to train the deep learning model of the diagnostic support system. However, cases in which multiple abnormal events occur must also be considered, for which obtaining training data becomes difficult due to the large number of combinations of possible abnormal events. This study proposes an approach to maintain diagnostic performance for multiple abnormal events by training a deep learning model with data on single abnormal events only. The proposed approach is applied to an existing algorithm that can perform feature selection and multi-label classification. We choose an extremely randomized trees classifier to select dedicated monitoring parameters for target abnormal events. In diagnosing each event occurrence independently, two-channel convolutional neural networks are employed as sub-models. The algorithm was tested in a case study with various scenarios, including single and multiple abnormal events. Results demonstrated that the proposed approach maintained diagnostic performance for 15 single abnormal events and significantly improved performance for 105 multiple abnormal events compared to the base model.

ADA: Advanced data analytics methods for abnormal frequent episodes in the baseline data of ISD

  • Biswajit Biswal;Andrew Duncan;Zaijing Sun
    • Nuclear Engineering and Technology
    • /
    • 제54권11호
    • /
    • pp.3996-4004
    • /
    • 2022
  • The data collected by the In-Situ Decommissioning (ISD) sensors are time-specific, age-specific, and developmental stage-specific. Research has been done on the stream data collected by ISD testbed in the recent few years to seek both frequent episodes and abnormal frequent episodes. Frequent episodes in the data stream have confirmed the daily cycle of the sensor responses and established sequences of different types of sensors, which was verified by the experimental setup of the ISD Sensor Network Test Bed. However, the discovery of abnormal frequent episodes remained a challenge because these abnormal frequent episodes are very small signals and may be buried in the background noise of voltage and current changes. In this work, we proposed Advanced Data Analytics (ADA) methods that are applied to the baseline data to identify frequent episodes and extended our approach by adding more features extracted from the baseline data to discover abnormal frequent episodes, which may lead to the early indicators of ISD system failures. In the study, we have evaluated our approach using the baseline data, and the performance evaluation results show that our approach is able to discover frequent episodes as well as abnormal frequent episodes conveniently.

Calculated Damage of Italian Ryegrass in Abnormal Climate Based World Meteorological Organization Approach Using Machine Learning

  • Jae Seong Choi;Ji Yung Kim;Moonju Kim;Kyung Il Sung;Byong Wan Kim
    • 한국초지조사료학회지
    • /
    • 제43권3호
    • /
    • pp.190-198
    • /
    • 2023
  • This study was conducted to calculate the damage of Italian ryegrass (IRG) by abnormal climate using machine learning and present the damage through the map. The IRG data collected 1,384. The climate data was collected from the Korea Meteorological Administration Meteorological data open portal.The machine learning model called xDeepFM was used to detect IRG damage. The damage was calculated using climate data from the Automated Synoptic Observing System (95 sites) by machine learning. The calculation of damage was the difference between the Dry matter yield (DMY)normal and DMYabnormal. The normal climate was set as the 40-year of climate data according to the year of IRG data (1986~2020). The level of abnormal climate was set as a multiple of the standard deviation applying the World Meteorological Organization (WMO) standard. The DMYnormal was ranged from 5,678 to 15,188 kg/ha. The damage of IRG differed according to region and level of abnormal climate with abnormal temperature, precipitation, and wind speed from -1,380 to 1,176, -3 to 2,465, and -830 to 962 kg/ha, respectively. The maximum damage was 1,176 kg/ha when the abnormal temperature was -2 level (+1.04℃), 2,465 kg/ha when the abnormal precipitation was all level and 962 kg/ha when the abnormal wind speed was -2 level (+1.60 ㎧). The damage calculated through the WMO method was presented as an map using QGIS. There was some blank area because there was no climate data. In order to calculate the damage of blank area, it would be possible to use the automatic weather system (AWS), which provides data from more sites than the automated synoptic observing system (ASOS).

준 지도 이상 탐지 기법의 성능 향상을 위한 섭동을 활용한 초구 기반 비정상 데이터 증강 기법 (Abnormal Data Augmentation Method Using Perturbation Based on Hypersphere for Semi-Supervised Anomaly Detection)

  • 정병길;권준형;민동준;이상근
    • 정보보호학회논문지
    • /
    • 제32권4호
    • /
    • pp.647-660
    • /
    • 2022
  • 최근 정상 데이터와 일부 비정상 데이터를 보유한 환경에서 딥러닝 기반 준 지도 학습 이상 탐지 기법이 매우 효과적으로 동작함이 알려져 있다. 하지만 사이버 보안 분야와 같이 실제 시스템에 대한 알려지지 않은 공격 등 비정상 데이터 확보가 어려운 환경에서는 비정상 데이터 부족이 발생할 가능성이 있다. 본 논문은 비정상 데이터가 정상 데이터보다 극히 작은 환경에서 준 지도 이상 탐지 기법에 적용 가능한 섭동을 활용한 초구 기반 비정상 데이터 증강 기법인 ADA-PH(Abnormal Data Augmentation Method using Perturbation based on Hypersphere)를 제안한다. ADA-PH는 정상 데이터를 잘 표현할 수 있는 초구의 중심으로부터 상대적으로 먼 거리에 위치한 샘플에 대해 적대적 섭동을 추가함으로써 비정상 데이터를 생성한다. 제안하는 기법은 비정상 데이터가 극소수로 존재하는 네트워크 침입 탐지 데이터셋에 대하여 데이터 증강을 수행하지 않았을 경우보다 평균적으로 23.63% 향상된 AUC가 도출되었고, 다른 증강 기법들과 비교했을 때 가장 높은 AUC가 또한 도출되었다. 또한, 실제 비정상 데이터에 유사한지에 대한 정량적 및 정성적 분석을 수행하였다.

Abnormal state diagnosis model tolerant to noise in plant data

  • Shin, Ji Hyeon;Kim, Jae Min;Lee, Seung Jun
    • Nuclear Engineering and Technology
    • /
    • 제53권4호
    • /
    • pp.1181-1188
    • /
    • 2021
  • When abnormal events occur in a nuclear power plant, operators must conduct appropriate abnormal operating procedures. It is burdensome though for operators to choose the appropriate procedure considering the numerous main plant parameters and hundreds of alarms that should be judged in a short time. Recently, various research has applied deep-learning algorithms to support this problem by classifying each abnormal condition with high accuracy. Most of these models are trained with simulator data because of a lack of plant data for abnormal states, and as such, developed models may not have tolerance for plant data in actual situations. In this study, two approaches are investigated for a deep-learning model trained with simulator data to overcome the performance degradation caused by noise in actual plant data. First, a preprocessing method using several filters was employed to smooth the test data noise, and second, a data augmentation method was applied to increase the acceptability of the untrained data. Results of this study confirm that the combination of these two approaches can enable high model performance even in the presence of noisy data as in real plants.

An Abnormal Breakpoint Data Positioning Method of Wireless Sensor Network Based on Signal Reconstruction

  • Zhijie Liu
    • Journal of Information Processing Systems
    • /
    • 제19권3호
    • /
    • pp.377-384
    • /
    • 2023
  • The existence of abnormal breakpoint data leads to poor channel balance in wireless sensor networks (WSN). To enhance the communication quality of WSNs, a method for positioning abnormal breakpoint data in WSNs on the basis of signal reconstruction is studied. The WSN signal is collected using compressed sensing theory; the common part of the associated data set is mined by exchanging common information among the cluster head nodes, and the independent parts are updated within each cluster head node. To solve the non-convergence problem in the distributed computing, the approximate term is introduced into the optimization objective function to make the sub-optimization problem strictly convex. And the decompressed sensing signal reconstruction problem is addressed by the alternating direction multiplier method to realize the distributed signal reconstruction of WSNs. Based on the reconstructed WSN signal, the abnormal breakpoint data is located according to the characteristic information of the cross-power spectrum. The proposed method can accurately acquire and reconstruct the signal, reduce the bit error rate during signal transmission, and enhance the communication quality of the experimental object.

병렬 오토인코더 기반의 비정상 신호 탐지 (Abnormal signal detection based on parallel autoencoders)

  • 이기배;이종현
    • 한국음향학회지
    • /
    • 제40권4호
    • /
    • pp.337-346
    • /
    • 2021
  • 일반적으로 비정상 신호 탐지 연구에서는 데이터 불균형으로 인해 정상 신호 특징을 주된 정보로 사용한다. 본 논문에서는 비정상 신호의 특징을 학습하는 병렬 오토인코더를 이용한 효율적인 비정상 신호 탐지기법을 제안한다. 제안된 동일한 구조로 이루어진 병렬 오토인코더는 정상 신호와 비정상 신호에 대한 특징을 학습하는 정상 복원기와 비정상 복원기로 구성되며, 정상 및 비정상 데이터를 순차적으로 학습함으로써 불균형 데이터 문제를 효율적으로 해결할 수 있다. 뿐만 아니라 보다 높은 탐지성능 향상을 위해서 부가적인 이진 분류기가 추가될 수 있다. 공개된 음향데이터를 이용한 실험결과, 제안된 병렬 탐지모델의 학습시간이 단일 오토인코더 탐지모델과 비교하여 약 1.31 ~ 1.61배 늘어나지만, 최소 22 % 이상의 Area Under Curve(AUC) 향상을 보였다. 또한, 사전에 훈련된 병렬 오토인코더를 이용하여 수중 음향데이터를 전이학습한 결과 수중 비정상 신호 AUC 탐지성능을 93 % 이상 향상시킬 수 있음을 확인하였다.

A Visualization Scheme with a Calendar Heat Map for Abnormal Pattern Analysis in the Manufacturing Process

  • Chankhihort, Doung;Lim, Byung-Muk;Lee, Gyu-Jung;Choi, Sungsu;Kwon, Sun-Ock;Lee, Sang-Hyun;Kang, Jeong-Tae;Nasridinov, Aziz;Yoo, Kwan-Hee
    • International Journal of Contents
    • /
    • 제13권2호
    • /
    • pp.21-28
    • /
    • 2017
  • Abnormal data in the manufacturing process makes it difficult to find useful information that can be applied in data management for the manufacturing industry. It causes various problems in the daily process of production. An issue from the abnormal data can be handled by our method that uses big data and visualization. Visualization is a new technology that transforms data representation into a two-dimensional representation. Nowadays, many newly developed technologies provide data analysis, algorithm, optimization, and high efficiency, and they meet user requirements. We propose combined production of the data visualization approach that uses integrative visualization of sources of abnormal pattern analysis results. The perceived idea of the proposed approach can solve the problem as it also works for big data. It can also improve the performance and understanding by using visualization and solving issues that occur in the manufacturing process with a calendar heat map.

랜덤포레스트를 이용한 기상 환경에 따른 이상기온 분류 (Classification Abnormal temperatures based on Meteorological Environment using Random forests)

  • 김윤수;송광윤;장인홍
    • 통합자연과학논문집
    • /
    • 제17권1호
    • /
    • pp.1-12
    • /
    • 2024
  • Many abnormal climate events are occurring around the world. The cause of abnormal climate is related to temperature. Factors that affect temperature include excessive emissions of carbon and greenhouse gases from a global perspective, and air circulation from a local perspective. Due to the air circulation, many abnormal climate phenomena such as abnormally high temperature and abnormally low temperature are occurring in certain areas, which can cause very serious human damage. Therefore, the problem of abnormal temperature should not be approached only as a case of climate change, but should be studied as a new category of climate crisis. In this study, we proposed a model for the classification of abnormal temperature using random forests based on various meteorological data such as longitudinal observations, yellow dust, ultraviolet radiation from 2018 to 2022 for each region in Korea. Here, the meteorological data had an imbalance problem, so the imbalance problem was solved by oversampling. As a result, we found that the variables affecting abnormal temperature are different in different regions. In particular, the central and southern regions are influenced by high pressure (Mainland China, Siberian high pressure, and North Pacific high pressure) due to their regional characteristics, so pressure-related variables had a significant impact on the classification of abnormal temperature. This suggests that a regional approach can be taken to predict abnormal temperatures from the surrounding meteorological environment. In addition, in the event of an abnormal temperature, it seems that it is possible to take preventive measures in advance according to regional characteristics.

Pre-Evaluation for Detecting Abnormal Users in Recommender System

  • Lee, Seok-Jun;Kim, Sun-Ok;Lee, Hee-Choon
    • Journal of the Korean Data and Information Science Society
    • /
    • 제18권3호
    • /
    • pp.619-628
    • /
    • 2007
  • This study is devoted to suggesting the norm of detection abnormal users who are inferior to the other users in the recommender system compared with estimation accuracy. To select the abnormal users, we propose the pre-filtering method by using the preference ratings to the item rated by users. In this study, the experimental result shows the possibility of detecting the abnormal users before the process of preference estimation through the prediction algorithm. And It will be possible to improve the performance of the recommender system by using this detecting norm.

  • PDF