• 제목/요약/키워드: removal of outliers

검색결과 21건 처리시간 0.026초

이상자료가 연안 환경자료의 통계 척도에 미치는 영향 (Impact of Outliers on the Statistical Measures of the Environmental Monitoring Data in Busan Coastal Sea)

  • 조홍연;이기섭;안순모
    • Ocean and Polar Research
    • /
    • 제38권2호
    • /
    • pp.149-159
    • /
    • 2016
  • The statistical measures of the coastal environmental data are used in a variety of statistical inferences, hypothesis tests, and data-driven modeling. If the measures are biased, then the statistical estimations and models may also be biased and this potential for bias is great when data contain some outliers defined as extraordinary large or small data values. This study aims to suggest more robust statistical measures as alternatives to more commonly used measures and to assess the performance these robust measures through a quantitative evaluation of more typical measures, such as in terms of locations, spreads, and shapes, with regard to environmental monitoring data in the Busan coastal sea. The detection of outliers within the data was carried out on the basis of Rosner's test. About 5-10% of the nutrient data were found to contain outliers based on Rosner's test. After removal (zero-weighting) of the outliers in the data sets, the relative change ratios of the mean and standard deviation between before and after outlier-removal conditions revealed the figures 13 and 33%, respectively. The variation magnitudes of skewness and kurtosis are 1.36 and 8.11 in a decreasing trend, respectively. On the other hand, the change ratios for more robust measures regarding the mean and standard deviation are 3.7-10.5%, and the variation magnitudes of robust skewness and kurtosis are about only 2-4% of the magnitude of the non-robust measures. The robust measures can be regarded as outlier-resistant statistical measures based on the relatively small changes in the scenarios before and after outlier removal conditions.

텐서보팅(Tensor Voting)기법을 이용한 지상라이다 자료의 노이즈 처리 (Noise Removal of Terrestrial LiDAR Data Using Tensor Voting Method)

  • 서일홍;손홍규;김창재;임진희
    • 한국측량학회:학술대회논문집
    • /
    • 한국측량학회 2010년 춘계학술발표회 논문집
    • /
    • pp.157-160
    • /
    • 2010
  • Terrestrial LiDAR data contains outliers which do not need in processing purpose. That is inefficient in the aspect of productivity. These noise requires manual process to be removed, which causes inefficiency in aspect of productivity. The purpose of this research is to demonstrate a possibility of automatic outlier removal of LiDAR data using 3D Tensor Voting method. For this, we presented in this article about the procedure to perform the application of Tensor Voting algorithm to the real data from terrestrial LiDAR.

  • PDF

TWR 기반 고정밀 측위를 위한 단일 이상측정치 제거 기술 (Single Outlier Removal Technology for TWR based High Precision Localization)

  • 이창은;성태경
    • 로봇학회논문지
    • /
    • 제12권3호
    • /
    • pp.350-355
    • /
    • 2017
  • UWB (Ultra Wide Band) refers to a system with a bandwidth of over 500 MHz or a bandwidth of 20% of the center frequency. It is robust against channel fading and has a wide signal bandwidth. Using the IR-UWB based ranging system, it is possible to obtain decimeter-level ranging accuracy. Furthermore, IR-UWB system enables acquisition over glass or cement with high resolution. In recent years, IR-UWB-based ranging chipsets have become cheap and popular, and it has become possible to implement positioning systems of several tens of centimeters. The system can be configured as one-way ranging (OWR) positioning system for fast ranging and TWR (two-way ranging) positioning system for cheap and robust ranging. On the other hand, the ranging based positioning system has a limitation on the number of terminals for localization because it takes time to perform a communication procedure to perform ranging. To overcome this problem, code multiplexing and channel multiplexing are performed. However, errors occur in measurement due to interference between channels and code, multipath, and so on. The measurement filtering is used to reduce the measurement error, but more fundamentally, techniques for removing these measurements should be studied. First, the TWR based positioning was analyzed from a stochastic point of view and the effects of outlier measurements were summarized. The positioning algorithm for analytically identifying and removing single outlier is summarized and extended to three dimensions. Through the simulation, we have verified the algorithm to detect and remove single outliers.

통행시간 추정을 위한 TCS 데이터의 전처리 모형 개발 (A Development of Preprocessing Models of Toll Collection System Data for Travel Time Estimation)

  • 이현석;남궁성
    • 한국ITS학회 논문지
    • /
    • 제8권5호
    • /
    • pp.1-11
    • /
    • 2009
  • TCS (Toll Collection System) 데이터는 원시 데이터 자체로서도 구간의 교통상황을 어느 정도 반영할 수 있는 교통특 성을 내포하고 있다. 그러나 TCS 데이터에는 이상치가 포함되어 있어 이러한 데이터는 해당 구간의 통행시간을 대표한다고 볼 수 없으므로 만약 이러한 이상치들이 포함되어 있음에도 불구하고 제거하지 않고 집락을 한다면 이상치들로 인해 통행시간은 크게 왜곡 될 가능성이 있다. 특히 장거리 구간일수록 통행시간의 분산이 증가하여 동일구간 동일시간대라도 다양한 통행시간이 분포하고 있다. 구간이 길어질수록 통행시간의 변동이 심하여 적절한 통행시간 대푯값을 구하기가 어렵다. 따라서 TCS 자료를 이용하여 통행시간의 대푯값을 산정하기 위해서는 통행시간의 변동 특성을 파악하는 것이 중요하다. 본 연구에서는 TCS 데이터의 전처리 기법을 개선하되 구간의 길이와 교통상황에 따른 통행시간의 변동을 고려하여 TCS 원시데이터로부터 시 공간적 통행패턴을 파악할 수 있는 의미 있는 통행시간을 추출하고자 한다.

  • PDF

Improved Lexicon-driven based Chord Symbol Recognition in Musical Images

  • Dinh, Cong Minh;Do, Luu Ngoc;Yang, Hyung-Jeong;Kim, Soo-Hyung;Lee, Guee-Sang
    • International Journal of Contents
    • /
    • 제12권4호
    • /
    • pp.53-61
    • /
    • 2016
  • Although extensively developed, optical music recognition systems have mostly focused on musical symbols (notes, rests, etc.), while disregarding the chord symbols. The process becomes difficult when the images are distorted or slurred, although this can be resolved using optical character recognition systems. Moreover, the appearance of outliers (lyrics, dynamics, etc.) increases the complexity of the chord recognition. Therefore, we propose a new approach addressing these issues. After binarization, un-distortion, and stave and lyric removal of a musical image, a rule-based method is applied to detect the potential regions of chord symbols. Next, a lexicon-driven approach is used to optimally and simultaneously separate and recognize characters. The score that is returned from the recognition process is used to detect the outliers. The effectiveness of our system is demonstrated through impressive accuracy of experimental results on two datasets having a variety of resolutions.

음성 특성 및 음성 독립 변수의 사상체질 분류로의 적용 방법 (Application of Vocal Properties and Vocal Independent Features to Classifying Sasang Constitution)

  • 김근호;강남식;구본초;김종열
    • 사상체질의학회지
    • /
    • 제23권4호
    • /
    • pp.458-470
    • /
    • 2011
  • 1. Objectives Vocal characteristics are commonly considered as an important factor in determining the Sasang constitution and the health condition. We have tried to find out the classification procedure to distinguish the constitution objectively and quantitatively by analyzing the characteristics of subject's voice without noise and error. 2. Methods In this study, we extract the vocal features from voice selected with prior information, remove outliers, minimize the correlated features, correct the features with normalization according to gender and age, and make the discriminant functions that are adaptive to gender and age from the features for improving diagnostic accuracy. 3. Results and Conclusions Finally, the discriminant functions produced about 45% accuracy to classify the constitution for every age interval and every gender, and the diagnostic accuracy was meaningful as the result from only the voice.

국내산 석회석의 비교숙련도 시험용 시료 제조 및 평가 (Preparation and evaluation of limestone reference material for a proficiency test)

  • 정충호;박덕원;김성민;유응철
    • 분석과학
    • /
    • 제22권1호
    • /
    • pp.82-91
    • /
    • 2009
  • 국내산 석회석을 이용하여 석회석 시료의 RRT 시험용 시료를 제조하여 XRF 및 습식 분석, ICP-OES를 이용한 기기 분석을 수행하였고 그 결과를 통계적 방법에 의하여 시료의 균질도를 평가하였다. 분석 결과 몇몇 시료의 경우 예상치 못했던 정규 분포로부터의 이상성이 발견되었으며 이상치를 제거한 후 측정한 모든 성분에 대하여 정규 분포 곡선에서 95% 신뢰 구간에서의 신뢰성 있는 표준 시료를 얻을 수 있었다.

Outlier 데이터 제거를 통한 미세먼지 예보성능의 향상 (Improvement of PM Forecasting Performance by Outlier Data Removing)

  • 전영태;유숙현;권희용
    • 한국멀티미디어학회논문지
    • /
    • 제23권6호
    • /
    • pp.747-755
    • /
    • 2020
  • In this paper, we deal with outlier data problems that occur when constructing a PM2.5 fine dust forecasting system using a neural network. In general, when learning a neural network, some of the data are not helpful for learning, but rather disturbing. Those are called outlier data. When they are included in the training data, various problems such as overfitting occur. In building a PM2.5 fine dust concentration forecasting system using neural network, we have found several outlier data in the training data. We, therefore, remove them, and then make learning 3 ways. Over_outlier model removes outlier data that target concentration is low, but the model forecast is high. Under_outlier model removes outliers data that target concentration is high, but the model forecast is low. All_outlier model removes both Over_outlier and Under_outlier data. We compare 3 models with a conventional outlier removal model and non-removal model. Our outlier removal model shows better performance than the others.

A NEW LANDSAT IMAGE CO-REGISTRATION AND OUTLIER REMOVAL TECHNIQUES

  • Kim, Jong-Hong;Heo, Joon;Sohn, Hong-Gyoo
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 2006년도 Proceedings of ISRS 2006 PORSEC Volume II
    • /
    • pp.594-597
    • /
    • 2006
  • Image co-registration is the process of overlaying two images of the same scene. One of which is a reference image, while the other (sensed image) is geometrically transformed to the one. Numerous methods were developed for the automated image co-registration and it is known as a time-consuming and/or computation-intensive procedure. In order to improve efficiency and effectiveness of the co-registration of satellite imagery, this paper proposes a pre-qualified area matching, which is composed of feature extraction with Laplacian filter and area matching algorithm using correlation coefficient. Moreover, to improve the accuracy of co-registration, the outliers in the initial matching point should be removed. For this, two outlier detection techniques of studentized residual and modified RANSAC algorithm are used in this study. Three pairs of Landsat images were used for performance test, and the results were compared and evaluated in terms of robustness and efficiency.

  • PDF

A New Landsat Image Co-Registration and Outlier Removal Techniques

  • Kim, Jong-Hong;Heo, Joon;Sohn, Hong-Gyoo
    • 대한원격탐사학회지
    • /
    • 제22권5호
    • /
    • pp.439-443
    • /
    • 2006
  • Image co-registration is the process of overlaying two images of the same scene. One of which is a reference image, while the other (sensed image) is geometrically transformed to the one. Numerous methods were developed for the automated image co-registration and it is known as a timeconsuming and/or computation-intensive procedure. In order to improve efficiency and effectiveness of the co-registration of satellite imagery, this paper proposes a pre-qualified area matching, which is composed of feature extraction with Laplacian filter and area matching algorithm using correlation coefficient. Moreover, to improve the accuracy of co-registration, the outliers in the initial matching point should be removed. For this, two outlier detection techniques of studentized residual and modified RANSAC algorithm are used in this study. Three pairs of Landsat images were used for performance test, and the results were compared and evaluated in terms of robustness and efficiency.