• Title/Summary/Keyword: Anomaly prediction

Search Result 105, Processing Time 0.027 seconds

Assessment of the Prediction Performance of Ensemble Size-Related in GloSea5 Hindcast Data (기상청 기후예측시스템(GloSea5)의 과거기후장 앙상블 확대에 따른 예측성능 평가)

  • Park, Yeon-Hee;Hyun, Yu-Kyung;Heo, Sol-Ip;Ji, Hee-Sook
    • Atmosphere
    • /
    • v.31 no.5
    • /
    • pp.511-523
    • /
    • 2021
  • This study explores the optimal ensemble size to improve the prediction performance of the Korea Meteorological Administration's operational climate prediction system, global seasonal forecast system version 5 (GloSea5). The GloSea5 produces an ensemble of hindcast data using the stochastic kinetic energy backscattering version2 (SKEB2) and timelagged ensemble. An experiment to increase the hindcast ensemble from 3 to 14 members for four initial dates was performed and the improvement and effect of the prediction performance considering Root Mean Square Error (RMSE), Anomaly Correlation Coefficient (ACC), ensemble spread, and Ratio of Predictable Components (RPC) were evaluated. As the ensemble size increased, the RMSE and ACC prediction performance improved and more significantly in the high variability area. In spread and RPC analysis, the prediction accuracy of the system improved as the ensemble size increased. The closer the initial date, the better the predictive performance. Results show that increasing the ensemble to an appropriate number considering the combination of initial times is efficient.

Two Overarching Teleconnection Mechanisms Affecting the Prediction of the 2018 Korean Heat Waves

  • Wie, Jieun;Moon, Byung-Kwon
    • Journal of the Korean earth science society
    • /
    • v.43 no.4
    • /
    • pp.511-519
    • /
    • 2022
  • Given the significant social and economic impact caused by heat waves, there is a pressing need to predict them with high accuracy and reliability. In this study, we analyzed the real-time forecast data from six models constituting the Subseasonal-to-Seasonal (S2S) prediction project, to elucidate the key mechanisms contributing to the prediction of the recent record-breaking Korean heat wave event in 2018. Weekly anomalies were first obtained by subtracting the 2017-2020 mean values for both S2S model simulations and observations. By comparing four Korean heat-wave-related indices from S2S models to the observed data, we aimed to identify key climate processes affecting prediction accuracy. The results showed that superior performance at predicting the 2018 Korean heat wave was achieved when the model showed better prediction performance for the anomalous anticyclonic activity in the upper troposphere of Eastern Europe and the cyclonic circulation over the Western North Pacific (WNP) region compared to the observed data. Furthermore, the development of upper-tropospheric anticyclones in Eastern Europe was closely related to global warming and the occurrence of La Niña events. The anomalous cyclonic flow in the WNP region coincided with enhancements in Madden-Julian oscillation phases 4-6. Our results indicate that, for the accurate prediction of heat waves, such as the 2018 Korean heat wave, it is imperative for the S2S models to realistically reproduce the variabilities over the Eastern Europe and WNP regions.

An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining (베이지안 확률 및 폐쇄 순차패턴 마이닝 방식을 이용한 설명가능한 로그 이상탐지 시스템)

  • Yun, Jiyoung;Shin, Gun-Yoon;Kim, Dong-Wook;Kim, Sang-Soo;Han, Myung-Mook
    • Journal of Internet Computing and Services
    • /
    • v.22 no.2
    • /
    • pp.77-87
    • /
    • 2021
  • With the development of the Internet and personal computers, various and complex attacks begin to emerge. As the attacks become more complex, signature-based detection become difficult. It leads to the research on behavior-based log anomaly detection. Recent work utilizes deep learning to learn the order and it shows good performance. Despite its good performance, it does not provide any explanation for prediction. The lack of explanation can occur difficulty of finding contamination of data or the vulnerability of the model itself. As a result, the users lose their reliability of the model. To address this problem, this work proposes an explainable log anomaly detection system. In this study, log parsing is the first to proceed. Afterward, sequential rules are extracted by Bayesian posterior probability. As a result, the "If condition then results, post-probability" type rule set is extracted. If the sample is matched to the ruleset, it is normal, otherwise, it is an anomaly. We utilize HDFS datasets for the experiment, resulting in F1score 92.7% in test dataset.

Manufacturing Data Preprocessing Method and Product Classification Method using FFT (FFT를 활용한 제조데이터 전처리 및 제품분류)

  • Kim, Han-sol;Jin, Kyo-hong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.10a
    • /
    • pp.82-84
    • /
    • 2021
  • Through the smart factory construction project, sensor data such as power, vibration, pressure, and temperature are collected from production facilities, and services such as predictive maintenance, defect prediction, and abnormality detection are developed through data analysis. In general, in the case of manufacturing data, because the imbalance between normal and abnormal data is extreme, an anomaly detection service is preferred. In this paper, FFT method is used to extract feature data of manufacturing data as a pre-stage of the anomaly detection service development. Using this method, we classified the produced products and confirmed results. In other words, after FFT of the representative pattern for each product, we verified whether product classification was possible or not, by calculating correlation coefficient.

  • PDF

Development and Assessment of Dynamical Seasonal Forecast System Using the Cryospheric Variables (빙권요소를 활용한 겨울철 역학 계절예측 시스템의 개발 및 검증)

  • Shim, Taehyoun;Jeong, Jee-Hoon;Ok, Jung;Jeong, Hyun-Sook;Kim, Baek-Min
    • Atmosphere
    • /
    • v.25 no.1
    • /
    • pp.155-167
    • /
    • 2015
  • A dynamical seasonal prediction system for boreal winter utilizing cryospheric information was developed. Using the Community Atmospheric Model, version3, (CAM3) as a modeling system, newly developed snow depth initialization method and sea ice concentration treatment were implemented to the seasonal prediction system. Daily snow depth analysis field was scaled in order to prevent climate drift problem before initializing model's snow fields and distributed to the model snow-depth layers. To maximize predictability gain from land surface, we applied one-month-long training procedure to the prediction system, which adjusts soil moisture and soil temperature to the imposed snow depth. The sea ice concentration over the Arctic region for prediction period was prescribed with an anomaly-persistent method that considers seasonality of sea ice. Ensemble hindcast experiments starting at 1st of November for the period 1999~2000 were performed and the predictability gain from the imposed cryospheric informations were tested. Large potential predictability gain from the snow information was obtained over large part of high-latitude and of mid-latitude land as a result of strengthened land-atmosphere interaction in the modeling system. Large-scale atmospheric circulation responses associated with the sea ice concentration anomalies were main contributor to the predictability gain.

Development of Dynamical Seasonal Prediction System for Northern Winter using the Cryospheric Condition of Late Autumn (가을철 빙권 조건을 활용한 겨울철 역학 계절 예측시스템의 개발)

  • Shim, Taehyoun;Jeong, Jee-Hoon;Kim, Baek-Min;Kim, Seong-Joong;Kim, Hyun-Kyung
    • Atmosphere
    • /
    • v.23 no.1
    • /
    • pp.73-83
    • /
    • 2013
  • In recent several years, East Asia, Europe and North America have suffered successive cold winters and a number of historical records on the extreme weathers are replaced with new record-breaking cold events. As a possible explanation, several studies suggested that cryospheric conditions of Northern Hemisphere (NH), i.e. Arctic sea-ice and snow cover over northern part of major continents, are changing significantly and now play an active role for modulating midlatitude atmospheric circulation patterns that could bring cold winters for some regions in midlatitude. In this study, a dynamical seasonal prediction system for NH winter is newly developed using the snow depth initialization technique and statistically predicted sea-ice boundary condition. Since the snow depth shows largest variability in October, entire period of October has been utilized as a training period for the land surface initialization and model land surface during the period is continuously forced by the observed daily atmospheric conditions and snow depths. A simple persistent anomaly decaying toward an averaged sea-ice condition has been used for the statistical prediction of sea-ice boundary conditions. The constructed dynamical prediction system has been tested for winter 2012/13 starting at November 1 using 16 different initial conditions and the results are discussed. Implications and a future direction for further development are also described.

Experimental verification for prediction method of anomaly ahead of tunnel face by using electrical resistivity tomography

  • Lee, Kang-Hyun;Park, Jin-Ho;Park, Jeongjun;Lee, In-Mo;Lee, Seok-Won
    • Geomechanics and Engineering
    • /
    • v.20 no.6
    • /
    • pp.475-484
    • /
    • 2020
  • The prediction of the ground conditions ahead of a tunnel face is very important, especially for tunnel boring machine (TBM) tunneling, because encountering unexpected anomalies during tunnel excavation can cause a considerable loss of time and money. Several prediction techniques, such as BEAM, TSP, and GPR, have been suggested. However, these methods have various shortcomings, such as low accuracy and low resolution. Most studies on electrical resistivity tomography surveys have been conducted using numerical simulation programs, but laboratory experiments were just a few. Furthermore, most studies of scaled model tests on electrical resistivity tomography were conducted only on the ground surface, which is a different environment as compared to that of mechanized tunneling. This study performed a laboratory experimental test to extend and verify a prediction method proposed by Lee et al., which used electrical resistivity tomography to predict the ground conditions ahead of a tunnel face in TBM tunneling environments. The results showed that the modified dipole-dipole array is better than the other arrays in terms of predicting the location and shape of the anomalies ahead of the tunnel face. Having longer upper and lower borehole lengths led to better accuracy of the survey. However, the number and length of boreholes should be properly controlled according to the field environments in practice. Finally, a modified and verified technique to predict the ground conditions ahead of a tunnel face during TBM tunneling is proposed.

Assessment of Performance on the Asian Dust Generation in Spring Using Hindcast Data in Asian Dust Seasonal Forecasting Model (황사장기예측자료를 이용한 봄철 황사 발생 예측 특성 분석)

  • Kang, Misun;Lee, Woojeong;Chang, Pil-Hun;Kim, Mi-Gyeong;Boo, Kyung-On
    • Atmosphere
    • /
    • v.32 no.2
    • /
    • pp.149-162
    • /
    • 2022
  • This study investigated the prediction skill of the Asian dust seasonal forecasting model (GloSea5-ADAM) on the Asian dust and meteorological variables related to the dust generation for the period of 1991~2016. Additionally, we evaluated the prediction skill of those variables depending on the combination of the initial dates in the sub-seasonal scale for the dust source region affecting South Korea. The Asian dust and meteorological variables (10 m wind speed, 1.5 m relative humidity, and 1.5 m air temperature) from GloSea5-ADAM were compared to that from Synoptic observation and European Centre for medium range weather forecasts reanalysis v5, respectively, based on Mean Bias Error (MBE), Root Mean Square Error (RMSE), and Anomaly Correlation Coefficient (ACC) as evaluation criteria. In general, the Asian dust and meteorological variables in the source region showed high ACC in the prediction scale within one month. For all variables, the use of the initial dates closest to the prediction month led to the best performances based on MBE, RMSE, and ACC, and the performances could be improved by adjusting the number of ensembles considering the combination of the initial date. ACC was as high as 0.4 in Spring when using the closest two initial dates. In particular, the GloSea5-ADAM shows the best performance of Asian dust generation with an ACC of 0.60 in the occurrence frequency of Asian dust in March when using the closest initial dates for initial conditions.

A Study on Efficient AI Model Drift Detection Methods for MLOps (MLOps를 위한 효율적인 AI 모델 드리프트 탐지방안 연구)

  • Ye-eun Lee;Tae-jin Lee
    • Journal of Internet Computing and Services
    • /
    • v.24 no.5
    • /
    • pp.17-27
    • /
    • 2023
  • Today, as AI (Artificial Intelligence) technology develops and its practicality increases, it is widely used in various application fields in real life. At this time, the AI model is basically learned based on various statistical properties of the learning data and then distributed to the system, but unexpected changes in the data in a rapidly changing data situation cause a decrease in the model's performance. In particular, as it becomes important to find drift signals of deployed models in order to respond to new and unknown attacks that are constantly created in the security field, the need for lifecycle management of the entire model is gradually emerging. In general, it can be detected through performance changes in the model's accuracy and error rate (loss), but there are limitations in the usage environment in that an actual label for the model prediction result is required, and the detection of the point where the actual drift occurs is uncertain. there is. This is because the model's error rate is greatly influenced by various external environmental factors, model selection and parameter settings, and new input data, so it is necessary to precisely determine when actual drift in the data occurs based only on the corresponding value. There are limits to this. Therefore, this paper proposes a method to detect when actual drift occurs through an Anomaly analysis technique based on XAI (eXplainable Artificial Intelligence). As a result of testing a classification model that detects DGA (Domain Generation Algorithm), anomaly scores were extracted through the SHAP(Shapley Additive exPlanations) Value of the data after distribution, and as a result, it was confirmed that efficient drift point detection was possible.

Regional anomalies of cosmic microwave background power spectrum

  • Ju, Young;Park, Chan-Gyung;Hwang, Jai-Chan
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.43 no.1
    • /
    • pp.38.4-39
    • /
    • 2018
  • We analyze the Planck 2015 cosmic microwave background temperature fluctuation data to find any anomaly in the angular power spectra measured for partial regions on the sky. For disks with radius of $20^{\circ}$, $45^{\circ}$ and $90^{\circ}$, which are densely overlapping on the sky, we estimate the power excess and its statistical significance relative to the LambdaCDM expectation for some chosen ranges of angular scales. We also investigate the dipolar asymmetry using the power excess maps obtained for some chosen angular scales, and confirm the previously announced consistent dipole directions. The average dipole amplitude and the inner products of dipoles have been measured from the power excess maps at different angular scales. We conclude that although dipole directions are consistent the measured amplitudes are not statistically significant compared to the LambdaCDM model prediction.

  • PDF