• Title/Summary/Keyword: Bayesian Rule

Search Result 95, Processing Time 0.021 seconds

An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining (베이지안 확률 및 폐쇄 순차패턴 마이닝 방식을 이용한 설명가능한 로그 이상탐지 시스템)

  • Yun, Jiyoung;Shin, Gun-Yoon;Kim, Dong-Wook;Kim, Sang-Soo;Han, Myung-Mook
    • Journal of Internet Computing and Services
    • /
    • v.22 no.2
    • /
    • pp.77-87
    • /
    • 2021
  • With the development of the Internet and personal computers, various and complex attacks begin to emerge. As the attacks become more complex, signature-based detection become difficult. It leads to the research on behavior-based log anomaly detection. Recent work utilizes deep learning to learn the order and it shows good performance. Despite its good performance, it does not provide any explanation for prediction. The lack of explanation can occur difficulty of finding contamination of data or the vulnerability of the model itself. As a result, the users lose their reliability of the model. To address this problem, this work proposes an explainable log anomaly detection system. In this study, log parsing is the first to proceed. Afterward, sequential rules are extracted by Bayesian posterior probability. As a result, the "If condition then results, post-probability" type rule set is extracted. If the sample is matched to the ruleset, it is normal, otherwise, it is an anomaly. We utilize HDFS datasets for the experiment, resulting in F1score 92.7% in test dataset.

Statistical Life Prediction of Corroded Pipeline Using Bayesian Inference (베이지안 추론법을 이용한 부식된 배관의 통계적 수명예측)

  • Noh, Yoojeong
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.4
    • /
    • pp.2401-2406
    • /
    • 2015
  • Pipelines are used by large heavy industries to deliver various types of fluids. Since this is important to maintain the performance of large systems, it is necessary to accurately predict remaining life of the corroded pipeline. However, predicting the remaining life is difficult due to uncertainties in the associated variables, such as geometries, material properties, corrosion rate, etc. In this paper, a statistical method for predicting corrosion remaining life is proposed using Bayesian inference. To accomplish this, pipeline failure probability was calculated using prior information about pipeline failure pressure according to elapsed time, and the given experimental data based on Bayes' rule. The corrosion remaining life was calculated as the elapsed time with 10 % failure probability. Using 10 and 50 samples generated from random variables affecting the corrosion of the pipe, the pipeline failure probability was estimated, after which the estimated remaining useful life was compared with the assumed true remaining useful life.

Analysis on Korean Economy with an Estimated DSGE Model after 2000 (DSGE 모형 추정을 이용한 2000년 이후 한국의 거시경제 분석)

  • Kim, Tae Bong
    • KDI Journal of Economic Policy
    • /
    • v.36 no.2
    • /
    • pp.1-64
    • /
    • 2014
  • This paper attempts to search the driving forces of the Korean economy after 2000 by analyzing an estimated DSGE model and observing the degree of implementation regarding non-systematic parts of both the monetary and fiscal policy during the global financial crisis. Two types of trends, various cyclical factors and frictions are introduced in the model for an empirical analysis in which historical decompositions of key macro variables are quantitatively assessed after 2000. While the monetary policy during the global financial crisis have reacted systematically in accordance with the estimated Taylor rule relatively, the fiscal policy which was aggressively expansionary is not fully explained by the estimated fiscal rule but more by the large magnitude of non-systematic reaction.

  • PDF

A Review of Machine Learning Algorithms for Fraud Detection in Credit Card Transaction

  • Lim, Kha Shing;Lee, Lam Hong;Sim, Yee-Wai
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.9
    • /
    • pp.31-40
    • /
    • 2021
  • The increasing number of credit card fraud cases has become a considerable problem since the past decades. This phenomenon is due to the expansion of new technologies, including the increased popularity and volume of online banking transactions and e-commerce. In order to address the problem of credit card fraud detection, a rule-based approach has been widely utilized to detect and guard against fraudulent activities. However, it requires huge computational power and high complexity in defining and building the rule base for pattern matching, in order to precisely identifying the fraud patterns. In addition, it does not come with intelligence and ability in predicting or analysing transaction data in looking for new fraud patterns and strategies. As such, Data Mining and Machine Learning algorithms are proposed to overcome the shortcomings in this paper. The aim of this paper is to highlight the important techniques and methodologies that are employed in fraud detection, while at the same time focusing on the existing literature. Methods such as Artificial Neural Networks (ANNs), Support Vector Machines (SVMs), naïve Bayesian, k-Nearest Neighbour (k-NN), Decision Tree and Frequent Pattern Mining algorithms are reviewed and evaluated for their performance in detecting fraudulent transaction.

Boundary Extraction Using Statistical Edge and Curvature Model

  • Park, Hae-Chul;Lee, J. S.;H. C. Shin;J. H. Cho;Kim, S. D.
    • Proceedings of the IEEK Conference
    • /
    • 2001.09a
    • /
    • pp.403-406
    • /
    • 2001
  • We propose an algorithm for extracting the boundary of an object. In order to take full advantage of global shape, our approach uses global shape parameters derived from Point Distribution Model (PDM). Unlike PDM, the proposed method models global shape using curvature as well as edge. The objective function of applying the shape model is formulated using Bayesian rule. We can extract the boundaries of an object by evaluating iteratively the solution maximizing the objective function. Experimental results show that the proposed method can reduce computation cost than the PDM and it is robust to noise, pose variation, and some occlusion.

  • PDF

Transformation of Mass Function and Joint Mass Function for Evidence Theory

  • Suh, Doug. Y.;Esogbue, Augustine O.
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.1 no.2
    • /
    • pp.16-34
    • /
    • 1991
  • It has been widely accepted that expert systems must reason from multiple sources of information that is to some degree evidential - uncertain, imprecise, and occasionally inaccurate - called evidential information. Evidence theory (Dempster/Shafet theory) provides one of the most general framework for representing evidential information compared to its alternatives such as Bayesian theory or fuzzy set theory. Many expert system applications require evidence to be specified in the continuous domain - such as time, distance, or sensor measurements. However, the existing evidence theory does not provide an effective approach for dealing with evidence about continuous variables. As an extension to Strat's pioneeiring work, this paper provides a new combination rule, a new method for mass function transffrmation, and a new method for rendering joint mass fuctions which are of great utility in evidence theory in the continuous domain.

  • PDF

Convergence Diagnostics for the Gibbs Sampler

  • Sohn, Joong-Kweon;Kim, Heon-Joo;Kang, Sang-Gil
    • Journal of the Korean Data and Information Science Society
    • /
    • v.7 no.1
    • /
    • pp.1-12
    • /
    • 1996
  • The Gibbs sampler is a substantially powerful tool in Bayesian analysis. However, it is necerssary to choose the numbert of iterations and the size of random samples. This problem has been studied by many researchers. The proposed procedures by them are generally difficult to apply to a practical problem. The attraction of the sampling based approaches is their conceptual simplicity and ease of implementation for users with available computing resources but without numerical analytic efforts. In this paper we consider the problem of determining the number of iterations t, which is simple to application.

  • PDF

K-means Clustering for Environmental Indicator Survey Data

  • Park, Hee-Chang;Cho, Kwang-Hyun
    • 한국데이터정보과학회:학술대회논문집
    • /
    • 2005.04a
    • /
    • pp.185-192
    • /
    • 2005
  • There are many data mining techniques such as association rule, decision tree, neural network analysis, clustering, genetic algorithm, bayesian network, memory-based reasoning, etc. We analyze 2003 Gyeongnam social indicator survey data using k-means clustering technique for environmental information. Clustering is the process of grouping the data into clusters so that objects within a cluster have high similarity in comparison to one another. In this paper, we used k-means clustering of several clustering techniques. The k-means clustering is classified as a partitional clustering method. We can apply k-means clustering outputs to environmental preservation and environmental improvement.

  • PDF

Bayesian Learning based Fuzzy Rule Extraction for Clustering (군집화를 위한 베이지안 학습 기반의 퍼지 규칙 추출)

  • 한진우;전성해;오경환
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2003.04c
    • /
    • pp.389-391
    • /
    • 2003
  • 컴퓨터 학습의 군집화는 주어진 데이터를 서로 유사한 몇 개의 집단으로 묶는 작업을 수행한다. 군집화를 위한 유사도 결정을 위한 측도는 많은 기법들에서 매우 다양한 측도들이 사용되고 또한 연구되어 왔다. 하지만 군집화의 결과에 대한 성능측정에 대한 객관적인 기준 설정이 어렵기 때문에 군집화 결과에 대한 해석은 매우 주관적이고 애매한 경우가 많다. 퍼지 군집화는 이러한 애매한 군집화 문제에 있어서 융통성 있는 군집 결정 방안을 제시해 준다. 각 개체들이 특정 군집에 속하게 될 퍼지 멤버 함수값을 원소로 하는 유사도 행렬을 통하여 군집화를 수행한다. 본 논문에서는 베이지안 학습을 통하여 군집화를 위한 퍼지 멤버 함수값을 구하였다. 본 연구에서는 최적의 퍼지 군집화 수행을 위하여 베이지안 학습 기반의 퍼지 규칙을 추출하였다. 인공적으로 만든 데이터와 기존의 기계 학습 데이터를 이용한 실험을 통하여 제안 방법의 성능을 확인하였다.

  • PDF

Spam-mail Filtering System Using Naive Bayesian Classifier and Message Rule (나이브 베이지안 분류자와 메세지 규칙을 이용한 스팸메일 필터링 시스템)

  • 조한철;조근식
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2002.04b
    • /
    • pp.223-225
    • /
    • 2002
  • 인터넷의 급속한 성장과 함께 E-Mail은 대표적인 통신수단의 하나가 되어버렸다. 편리하다는 점을 이용해서 엄청난 양의 스팸메일이 매일같이 쏟아져 오고 , 그 문제점의 심각성에 정보통신부에서 정보통신망 이용촉진 및 정보보호 등에 관한 법률이라는 새로운 법률까지 생겨났다. 본 논문에서는 이 법률에서 요구하는 '광고'라는 문구를 걸러내는 등의 메시지 규칙을 갖는 시스템과 기존의 문서 분류에 널리 쓰이던 나이브 베이지안 분류자(Naive Baesian Classifier)를 결합한 스팸 메일 필터링 시스템(Spam-mail Fitering System)을 제안한다. 제안된 시스템에서는 사용자가 직접 규칙을 작성할 필요없이 학습한 데이터를 갖고 자동으로 스팸메일을 분류할 수가 있다. 들어온 메일은 메시지 규칙 기반 필터가 먼저 적용되고, 메세지 규칙 기반 필터에서 분류되지 않으면 나이브 베이지안 필터에서 분류된다. 실험에서는 제안된 시스템의 성능을 평가하기 위해서 메시지 규칙을 사용한 시스템 및 나이브 베이지만 분류자 시스템과 비교 평가하였다. 또한 임계치를 변경함으로써 제안된 시스템의 성능을 높일 수있도록 하였다.

  • PDF