• 응용통계연구
• /
• 제30권1호
• /
• pp.159-167
• /
• 2017

일반적으로 잠재적 이상치군은 검정과정을 통해 최종적으로 이상치 여부를 판단하지만 검정절차를 생략하거나 모의실험에 의해 계산된 유의값을 기반으로 검정을 수행하는 이상치 탐지법들도 있다. 본 논문에서는 가면화나 수렁화현상을 피하기 위하여 이상치후보군에 속한 개별 관찰치를 검정하지 않고 이상치후보군의 부분집합들을 검정하는 절차를 제안한다. 제안된 방법의 활용을 보여주는 예제와 다른 방법과의 검정력 비교를 위한 모의실험 결과가 제시된다.

• ETRI Journal
• /
• 제41권4호
• /
• pp.437-451
• /
• 2019

Outlier detection techniques play an important role in enhancing the reliability of data communication in wireless sensor networks (WSNs). Considering the importance of outlier detection in WSNs, many outlier detection techniques have been proposed. Unfortunately, most of these techniques still have some potential limitations, that is, (a) high rate of false positives, (b) high time complexity, and (c) failure to detect outliers online. Moreover, these approaches mainly focus on either temporal outliers or spatial outliers. Therefore, this paper aims to introduce novel algorithms that successfully detect both temporal outliers and spatial outliers. Our contributions are twofold: (i) modifying the Hampel Identifier (HI) algorithm to achieve high accuracy identification rate in temporal outlier detection, (ii) combining the Gaussian process (GP) model and graph-based outlier detection technique to improve the performance of the algorithm in spatial outlier detection. The results demonstrate that our techniques outperform the state-of-the-art methods in terms of accuracy and work well with various data types.

• 응용통계연구
• /
• 제31권4호
• /
• pp.517-528
• /
• 2018

다변량 자료에서 특이점을 검출하고, 검출된 특이점을 시각화와 연결한 R 스크립트를 제공한다. 개발된 R 스크립트는 특이점을 검출하는 방법으로서 1) Robust Mahalanobis distance, 2) High Dimensional data, 3) Density-based approach 방법을 이용하였다. 특이점을 연결하면서 데이터 구조를 파악하기 위한 시각화 방법으로는 1) multidimensional scaling (MDS)와 minimal spanning tree (MST)를 K-means 군집분석과 연결하여 표시하는 방법, 2) MDS를 fviz cluster와 연결하는 방법, 3) principal component analysis (PCA)를 fviz cluster와 연결한 방법을 이용하였다. 사례분석의 예로서는 Major League Baseball (MLB) 자료에서 류현진이 적극적으로 활동하던 2013년, 2014년 투수자료를 이용하였다. 개발된 R 스트립트는 "http://www.knou.ac.kr/~sskim/ddpoutlier.html (R 스크립트와 R 패키지도 다운로드 받을 수 있다. 실행방법도 설명되어 있다.)"에서 다운받으면 된다.

• Wind and Structures
• /
• 제26권6호
• /
• pp.383-395
• /
• 2018

An accurate determination of wind speed distribution is the basis for an evaluation of the wind energy potential required to design a wind turbine, so it is important to estimate unknown parameters of wind speed distribution. In this paper, Gumbel distribution is used in modelling wind speed data, and alternative robust estimation methods to estimate its parameters are considered. The methodologies used to obtain the estimators of the parameters are least absolute deviation, weighted least absolute deviation, median/MAD and least median of squares. The performances of the estimators are compared with traditional estimation methods (i.e., maximum likelihood and least squares) according to bias, mean square deviation and total mean square deviation criteria using a Monte-Carlo simulation study for the data with and without outliers. The simulation results show that least median of squares and median/MAD estimators are more efficient than others for data with outliers in many cases. However, median/MAD estimator is not consistent for location parameter of Gumbel distribution in all cases. In real data application, it is firstly demonstrated that Gumbel distribution fits the daily mean wind speed data well and is also better one to model the data than Weibull distribution with respect to the root mean square error and coefficient of determination criteria. Next, the wind data modified by outliers is analysed to show the performance of the proposed estimators by using numerical and graphical methods.

• Ocean and Polar Research
• /
• 제38권2호
• /
• pp.149-159
• /
• 2016

The statistical measures of the coastal environmental data are used in a variety of statistical inferences, hypothesis tests, and data-driven modeling. If the measures are biased, then the statistical estimations and models may also be biased and this potential for bias is great when data contain some outliers defined as extraordinary large or small data values. This study aims to suggest more robust statistical measures as alternatives to more commonly used measures and to assess the performance these robust measures through a quantitative evaluation of more typical measures, such as in terms of locations, spreads, and shapes, with regard to environmental monitoring data in the Busan coastal sea. The detection of outliers within the data was carried out on the basis of Rosner's test. About 5-10% of the nutrient data were found to contain outliers based on Rosner's test. After removal (zero-weighting) of the outliers in the data sets, the relative change ratios of the mean and standard deviation between before and after outlier-removal conditions revealed the figures 13 and 33%, respectively. The variation magnitudes of skewness and kurtosis are 1.36 and 8.11 in a decreasing trend, respectively. On the other hand, the change ratios for more robust measures regarding the mean and standard deviation are 3.7-10.5%, and the variation magnitudes of robust skewness and kurtosis are about only 2-4% of the magnitude of the non-robust measures. The robust measures can be regarded as outlier-resistant statistical measures based on the relatively small changes in the scenarios before and after outlier removal conditions.

• 스마트미디어저널
• /
• 제13권4호
• /
• pp.48-56
• /
• 2024

축산업에서 환경의 이상치 탐지와 데이터 예측은 매우 중요한 과제이다. 대부분 시계열 데이터로 수집되는 축산 환경 데이터의 이상치는 급격한 생육환경의 변화와 예상치 못한 전염병의 징후를 나타낼 수 있으므로 이상치를 빠르게 탐지하는 것이 중요하다. 이상치의 빠른 탐지와 효과적인 대응은 가축의 스트레스를 최소화하고 전염병 발생 환경을 조기에 발견하여 농가의 경제적인 손실을 감소시키는 역할을 할 수 있다. 본 연구에서는 축산환경 데이터의 이상치 탐지 분야에서 이상치를 규정하는 경계값(Threshold) 설정에서 두 가지 설정 방법을 이용하여 실험하고 성능을 비교하였다. Mean Squared Error(MSE)를 활용한 이상치 탐지 방법과 Dynamic Threshold를 이용한 이상치 탐지 방법을 이용하여 이를 통해 주어진 이전 데이터의 평균값과의 변동성을 분석하여 이상 상황을 식별하는 연구를 진행하였다. MSE를 활용한 이상치 탐지 방법은 94.98% 정확도를 보였고 표준편차를 활용한 Dynamic Threshold 방법은 99.66%정확도로 성능이 더 우수함을 확인할 수 있었다.

• 응용통계연구
• /
• 제25권1호
• /
• pp.205-213
• /
• 2012

선형회귀모형에서 자료를 모형에 적합시킬 때 일반적으로 반응변수 변환을 시도하지만 적절한 변환함수의 결정은 몇개의 이상치들에 민감하게 반응한다는 것이 잘 알려져 있다. 이에 따라 이상치에 영향을 받지 않는 변수변환 방법들이 연구, 개발되고 있으나 최근에 Cheng (2005)에 의해 최소절사제곱추정치에 기반을 둔 절사 우도추정치 방법처럼 이상치의 숫자를 미리 정해야한다거나 많은 계산량이 필요하다는 단점들을 갖고 있다. 본 논문에서는 그와 같은 문제점을 해결하고 추정치의 강건성을 개선하는 새로운 방법을 제안하며 제안된 방법에서는 반응변수 변환에 따른 이상치 탐색법에 있어서 Hadi와 Simonoff (1993)가 제시한 단계적 절차를 응용, 적용한다.

• 한국수자원학회:학술대회논문집
• /
• 한국수자원학회 2018년도 학술발표회
• /
• pp.276-276
• /
• 2018

In this paper the potential of the Principle Component Analysis(PCA) technique that can be used to detect leaks in water pipe network blocks was evaluated. For this purpose the PCA was conducted to evaluate the relevance of the calculated outliers of a PCA model utilizing the recorded pipe flows and the recorded pipe leak incidents of a case study water distribution system. The PCA technique was enhanced by applying the computational algorithms developed in this study. The algorithms were designed to extract a partial set of flow data from the original 24 hour flow data so that the variability of the flows in the determined partial data set are minimal. The relevance of the calculated outliers of a PCA model and the recorded pipe leak incidents was analyzed. The results showed that the effectiveness of detecting leaks may improve by applying the developed algorithm. However, the analysis suggested that further development on the algorithm is needed to enhance the applicability of the PCA in detecting leaks in real-world water pipe networks.

• Asia Marketing Journal
• /
• 제16권3호
• /
• pp.77-100
• /
• 2014

Many researchers have analyzed the relationship between the financial success patterns of a motion picture and many other factors, such as the production cost, marketing, stars, awards, reviews, genre, and rating. Through these studies, many researchers and investors concluded that big budgets to make a blockbuster movie can serve as an insurance policy to meet their ROI; thus the box office is dominated by blockbuster movies. High-budget blockbuster movies are more likely to receive attention because these movies are more recognizable given their high expenses for production and casting. Therefore, audiences choose blockbusters in an effort to reduce the searching cost and to mitigate the possibility of a regrettable choice. This behavior of consumers, in turn, causes distributors to allocate screens for blockbusters, resulting in "concentration of blockbuster consumption." As such, low-budget films cannot easily become popular due to the lack of distribution. Indeed, low-budget films released on a small number of screens often end up becoming dismal failures. However, there are exceptional examples which are contrary to the general idea in the movie industry that a big budget and showings on a large number of screens can guarantee the success of a movie. Although researchers have attempted to analyze the performances of movies with small budgets, such movies are likely to be regarded as outliers and then be entirely discarded, as they are far from the 'three-sigma' range, especially given that previous research methodologies could not explain the financial success of such unique examples. This study attempts to explain the financial success at the box office of low-budget movies by applying the concept of the tunnel effect in quantum mechanics, as the phenomenon found in the movie industry is similar to a particle's movement in quantum physics. The tunneling effect is a phenomenon by which a particle without enough energy to pass over a potential barrier tunnels through it. Adopting the analogy, this study draws a tunneling probability function and cultural constant to forecast other outliers using the Schrödinger equation. Moreover, the study finds that word-of-mouth creates in the movie industry this phenomenon of finding outliers.

• 응용통계연구
• /
• 제31권4호
• /
• pp.463-473
• /
• 2018

연구에서는 선형회귀모형을 가정한 대형 데이터에서의 변수선택 알고리즘을 다룬다. 방법의 속도와 강건성에 주안점을 둔 여러 알고리즘들이 제안되었다. 그 중에서 streamwise 회귀 접근법을 사용한 VIF회귀는 신속하고 정확하게 수행된다. 그러나 VIF회귀는 최소제곱방법에 의해 모형이 추정되므로 이상치에 민감하다. 변수선택방법의 강건성을 높이기 위해 가중 추정치를 사용한 강건측도가 제안되었으며 강건 VIF회귀도 제안되었다. 본 연구에서는 잠재적 이상치를 탐지하여 제거한 후 VIF회귀를 수행하는, 빠르고 강건한 변수선택 방법을 제안한다. 제안된 방법은 모의실험과 데이터 분석 통해 다른 방법들과 비교된다.