• Title/Summary/Keyword: nonparametric detection

Search Result 47, Processing Time 0.018 seconds

Network Anomaly Detection using Hybrid Feature Selection

  • Kim Eun-Hye;Kim Se-Hun
    • Proceedings of the Korea Institutes of Information Security and Cryptology Conference
    • /
    • 2006.06a
    • /
    • pp.649-653
    • /
    • 2006
  • In this paper, we propose a hybrid feature extraction method in which Principal Components Analysis is combined with optimized k-Means clustering technique. Our approach hierarchically reduces the redundancy of features with high explanation in principal components analysis for choosing a good subset of features critical to improve the performance of classifiers. Based on this result, we evaluate the performance of intrusion detection by using Support Vector Machine and a nonparametric approach based on k-Nearest Neighbor over data sets with reduced features. The Experiment results with KDD Cup 1999 dataset show several advantages in terms of computational complexity and our method achieves significant detection rate which shows possibility of detecting successfully attacks.

  • PDF

Effects of the Reference Sample Size on the Performance of the Two-Sample Rank Detector (두 표본 순위 검파에서 기준 표본 크기가 검파기 성능에 미치는 영향)

  • Bae, Jinsoo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.8
    • /
    • pp.1515-1517
    • /
    • 2015
  • The effects of the reference sample size on the detection probability of the two-sample rank detector is investigated in this paper. The larger reference sample size shows the better performance of the detector. The effect is also shown to be saturated as the reference sample size becomes larger.

Statistical Tests for Edg Detection (에지 검출을 위한 통계적 검정법)

  • Im, Dong-Hun;Seong, Sin-Hui
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.3
    • /
    • pp.1021-1024
    • /
    • 2000
  • In this paper we describe a nonparametric Wilcoxon test and a parametric Z test based on statistical hypothesis testing for the detection of edges. We use the threshold determined by specifying significance level $\alpha$, while Bovik, Huang and Munson[4] consider the range of possible values of test statistics for the threshold. From the experimental results of edge detection, the Z method performs sensitively to the noisy image, while the Wilcoxon method is robust over both noisy nd noise-free images. Comparison with our statistical tests and Sobel operator shows that our tests perform more effectively in both noisy and noise-free images.

  • PDF

Simultaneous outlier detection and variable selection via difference-based regression model and stochastic search variable selection

  • Park, Jong Suk;Park, Chun Gun;Lee, Kyeong Eun
    • Communications for Statistical Applications and Methods
    • /
    • v.26 no.2
    • /
    • pp.149-161
    • /
    • 2019
  • In this article, we suggest the following approaches to simultaneous variable selection and outlier detection. First, we determine possible candidates for outliers using properties of an intercept estimator in a difference-based regression model, and the information of outliers is reflected in the multiple regression model adding mean shift parameters. Second, we select the best model from the model including the outlier candidates as predictors using stochastic search variable selection. Finally, we evaluate our method using simulations and real data analysis to yield promising results. In addition, we need to develop our method to make robust estimates. We will also to the nonparametric regression model for simultaneous outlier detection and variable selection.

Correlation Coefficients between Parametric and onparametric Test Statistics for Signal Detection Problems (신호 검파 문제에 쓰는 모수와 비모수 검정 통계량 사이의 상관계수)

  • Park So Ryoung;Kwon Hyoungmoon;Bae Jinsoo;Choi Sang Won;Lee Jumi;Song Iickho
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.30 no.6C
    • /
    • pp.541-550
    • /
    • 2005
  • In this paper, we address the derivation of joint distributions and correlation coefficients for four pairs of statistics used commonly in a number of signal detection schemes. The upper and lower bounds of the correlation coefficients are obtained, and interesting relationships between the correlation coefficients are derived. Explicit values of the correlation coefficients are given in the form of tables and figures for easy reference. The results in this paper should be useful in comparing various detection statistics.

Detection of Change-Points by Local Linear Regression Fit;

  • Kim, Jong Tae;Choi, Hyemi;Huh, Jib
    • Communications for Statistical Applications and Methods
    • /
    • v.10 no.1
    • /
    • pp.31-38
    • /
    • 2003
  • A simple method is proposed to detect the number of change points and test the location and size of multiple change points with jump discontinuities in an otherwise smooth regression model. The proposed estimators are based on a local linear regression fit by the comparison of left and right one-side kernel smoother. Our proposed methodology is explained and applied to real data and simulated data.

Nonparametric detection algorithm of discontinuity points in the variance function

  • Huh, Jib
    • Journal of the Korean Data and Information Science Society
    • /
    • v.18 no.3
    • /
    • pp.669-678
    • /
    • 2007
  • An algorithm to detect the number of discontinuity points of the variance function in regression model is proposed. The proposed algorithm is based on the left and right one-sided kernel estimators of the second moment function and test statistics of the existence of a discontinuity point coming from the asymptotic distribution of the estimated jump size. The finite sample performance is illustrated by simulated example.

  • PDF

Edge Detection using Statistical Hypothesis Testing

  • Lim, Dong-Hoon;Sung, Sin-Hee
    • Communications for Statistical Applications and Methods
    • /
    • v.6 no.3
    • /
    • pp.893-900
    • /
    • 1999
  • We use statistical tests which are useful for two-sample problem for detecting edges in gray-level images. An edge is detected by examining changes in gray-level value between adjacent pixel neighborhoods. Some experimental results show that nonparametric detectors such as Mann-Whitney test median test and Kolmogorov-Smirnov test perform effectively in both noisy and noise-free images while parametric T test is sensitive to noise.

  • PDF

Locating and identifying model-free structural nonlinearities and systems using incomplete measured structural responses

  • Liu, Lijun;Lei, Ying;He, Mingyu
    • Smart Structures and Systems
    • /
    • v.15 no.2
    • /
    • pp.409-424
    • /
    • 2015
  • Structural nonlinearity is a common phenomenon encountered in engineering structures under severe dynamic loading. It is necessary to localize and identify structural nonlinearities using structural dynamic measurements for damage detection and performance evaluation of structures. However, identification of nonlinear structural systems is a difficult task, especially when proper mathematical models for structural nonlinear behaviors are not available. In prior studies on nonparametric identification of nonlinear structures, the locations of structural nonlinearities are usually assumed known and all structural responses are measured. In this paper, an identification algorithm is proposed for locating and identifying model-free structural nonlinearities and systems using incomplete measurements of structural responses. First, equivalent linear structural systems are established and identified by the extended Kalman filter (EKF). The locations of structural nonlinearities are identified. Then, the model-free structural nonlinear restoring forces are approximated by power series polynomial models. The unscented Kalman filter (UKF) is utilized to identify structural nonlinear restoring forces and structural systems. Both numerical simulation examples and experimental test of a multi-story shear building with a MR damper are used to validate the proposed algorithm.

NONPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATION OF A CONCAVE RECEIVER OPERATING CHARACTERISTIC CURVE VIA GEOMETRIC PROGRAMMING

  • Lee, Kyeong-Eun;Lim, Johan
    • Bulletin of the Korean Mathematical Society
    • /
    • v.48 no.3
    • /
    • pp.523-537
    • /
    • 2011
  • A receiver operating characteristic (ROC) curve plots the true positive rate of a classier against its false positive rate, both of which are accuracy measures of the classier. The ROC curve has several interesting geometrical properties, including concavity which is a necessary condition for a classier to be optimal. In this paper, we study the nonparametric maximum likelihood estimator (NPMLE) of a concave ROC curve and its modification to reduce bias. We characterize the NPMLE as a solution to a geometric programming, a special type of a mathematical optimization problem. We find that the NPMLE is close to the convex hull of the empirical ROC curve and, thus, has smaller variance but positive bias at a given false positive rate. To reduce the bias, we propose a modification of the NPMLE which minimizes the $L_1$ distance from the empirical ROC curve. We numerically compare the finite sample performance of three estimators, the empirical ROC curve, the NMPLE, and the modified NPMLE. Finally, we apply the estimators to estimating the optimal ROC curve of the variance-threshold classier to segment a low depth of field image and to finding a diagnostic tool with multiple tests for detection of hemophilia A carrier.