• 응용통계연구
• /
• 제24권6호
• /
• pp.1249-1257
• /
• 2011

비모수적 결측치 대치법인 k-최근접 이웃(k-Nearest Neighbors; KNN) 대치법을 개선한 적응 최근접 이웃(Adaptive Nearest Neighbor; ANN) 대치법과 순차 k-최근접 이웃(Sequential k-Nearest Neighbor; SKNN) 대치법의 장점들을 결합한 순차 적응 최근접 이웃(Sequential Adaptive Nearest Neighbor; SANN) 대치법을 제안하고자 한다. 이 방법은 ANN 대치법의 장점인 자료의 국소적 특징을 반영할 뿐 아니라, SKNN 대치법과 같이 결측값 대치가 이루어진 개체를 다음 결측값을 대치할 때 사용함으로써 효율성에 개선이 있을 것으로 기대한다.

• 한국정보과학회논문지:소프트웨어및응용
• /
• 제36권4호
• /
• pp.273-282
• /
• 2009

소프트웨어 프로젝트 데이터를 이용한 각종 분석 예측 모델 생성시 직면하는 문제 중 하나는 데이터에 포함된 결측값이며 이에 대한 효과적인 방안은 결측값 대치 법이다. 대표적인 결측값 대치법인 K 최근접 이웃 대치법은 대치과정에서 결측값을 포함하는 인스턴스의 관측정보를 활용하지 못한다는 단점이 있다. 본 연구에서는 이러한 단점을 극복하기 위해 K 최근접 이웃 대치법과 최대 우도 추정법을 결합한 새로운 소프트웨어 프로젝트 수치 데이터용 결측값 대치법을 제안한다. 또한 결측값 대치법의 정확도를 비교하기 위한 새로운 측도를 함께 제안한다.

• 응용통계연구
• /
• 제28권1호
• /
• pp.23-31
• /
• 2015

통계적 분석을 할 때 결측치가 발생하는 것은 매우 통상적이다. 이러한 결측치를 대치하는 방법은 여러가지가 있으며, 기존에 사용되는 단일대치법으로 k-nearest neighbor(KNN) 방법이 있다. 하지만 KNN 방법은 k개의 최근접 이웃들 중 극단치나 이상치가 있을 때 편의를 일으킬 수 있다. 본 논문에서는 KNN 방법의 단점을 보완하여 가중 k-최근접이웃(Weighted k-Nearest Neighbors; WKNN) 대치법을 제안하였다. 또한 모의실험을 통해서 기존의 방법과 비교하였다.

• 문화기술의 융합
• /
• 제10권1호
• /
• pp.603-608
• /
• 2024

무응답 및 결측값은 표본 탈락, 설문조사에 대한 답변 회피 등으로 발생하며 정보의 손실 및 편향된 추론의 가능성이 있는 문제가 발생하게 되며, 이 경우 결측값을 적절한 값으로 바꾸는 대체가 필요하게 된다. 본 논문에서는 결측값에 대한 대체 방법으로 제안되었던 평균 대체, 다중회귀 대체, 랜덤 포레스트 대체, K-최근접 이웃 대체, 그리고 딥러닝을 기본으로 한 오토인코더 대체와 잡음제거 오토인코더 대체 방법을 비교한다. 결측값을 대체하는 이러한 방법들에 대해 설명하고, 연속형의 모의실험 데이터와 실제 데이터에 접목시켜 각 방법들을 비교하였다. 비교 결과 대부분의 경우에서 다중 대체 방법인 랜덤 포레스트 대체 방법과 잡음제거 오토인코더 대체 방법의 성능이 좋았음을 확인하였다.

• Communications for Statistical Applications and Methods
• /
• 제30권3호
• /
• pp.331-341
• /
• 2023

Handling missing values in data analysis is essential in constructing a good prediction model. The easiest way to handle missing values is to use complete case data, but this can lead to information loss within the data and invalid conclusions in data analysis. Imputation is a technique that replaces missing data with alternative values obtained from information in a dataset. Conventional imputation methods include K-nearest-neighbor imputation and multiple imputations. Recent methods include missForest, missRanger, and mixgb ,all which use machine learning algorithms. This paper compares the imputation techniques for datasets with mixed datatypes in various situations, such as data size, missing ratios, and missing mechanisms. To evaluate the performance of each method in mixed datasets, we propose a new imputation performance measure (IPM) that is a unified measurement applicable to numerical and categorical variables. We believe this metric can help find the best imputation method. Finally, we summarize the comparison results with imputation performances and computational times.

• Journal of the Korean Data and Information Science Society
• /
• 제17권2호
• /
• pp.421-436
• /
• 2006

Huang proposed a grey-based nearest neighbor approach to predict accurately missing attribute value in 2004. Our study proposes which way to decide the number of nearest neighbors using not only the deng's grey relational grade but also the wen's grey relational grade. Besides, our study uses not an arithmetic(unweighted) mean but a weighted one. Also, GRG is used by a weighted value when we impute missing values. There are four different methods - DU, DW, WU, WW. The performance of WW(Wen's GRG & weighted mean) method is the best of any other methods. It had been proven by Huang that his method was much better than mean imputation method and multiple imputation method. The performance of our study is far superior to that of Huang.

• 한국데이터정보과학회:학술대회논문집
• /
• 한국데이터정보과학회 2006년도 PROCEEDINGS OF JOINT CONFERENCEOF KDISS AND KDAS
• /
• pp.55-62
• /
• 2006

Huang proposed a grey-based nearest neighbor approach to predict accurately missing attribute value in 2004. Our study proposes which way to decide the number of nearest neighbors using not only the dong's grey relational grade but also the wen's grey relational grade. Besides, our study uses not an arithmetic(unweighted) mean but a weighted one. Also, GRG is used by a weighted value when we impute a missing values. There are four different methods - DU, DW, WU, WW. The performance of WW(wen's GRG & weighted mean) method is the best of my other methods. It had been proven by Huang that his method was much better than mean imputation method and multiple imputation method. The performance of our study is far superior to that of Huang.

• Journal of the Korean Data and Information Science Society
• /
• 제18권3호
• /
• pp.585-598
• /
• 2007

It is so difficult to get complete data when we conduct a questionaire in actuality. And we get inefficient results if we analyze statistical tests with ignoring missing values. Therefore, we use imputation methods which evaluate quality of data. This study proposes a imputation method by decomposition and voting with ordinal data. First, data are sorted by each variable. After that, imputation methods are used by each decomposition level. And the last step is selection of values with voting. The proposed method is evaluated by accuracy and RMSE. In conclusion, missing values are related to each variable, median imputation method using decomposition and voting is powerful.

• Journal of Information Processing Systems
• /
• 제10권3호
• /
• pp.395-411
• /
• 2014

Temporal medical data is often collected during patient treatments that require personal analysis. Each observation recorded in the temporal medical data is associated with measurements and time treatments. A major problem in the analysis of temporal medical data are the missing values that are caused, for example, by patients dropping out of a study before completion. Therefore, the imputation of missing data is an important step during pre-processing and can provide useful information before the data is mined. For each patient and each variable, this imputation replaces the missing data with a value drawn from an estimated distribution of that variable. In this paper, we propose a new method, called Newton's finite divided difference polynomial interpolation with condition order degree, for dealing with missing values in temporal medical data related to obesity. We compared the new imputation method with three existing subspace estimation techniques, including the k-nearest neighbor, local least squares, and natural cubic spline approaches. The performance of each approach was then evaluated by using the normalized root mean square error and the statistically significant test results. The experimental results have demonstrated that the proposed method provides the best fit with the smallest error and is more accurate than the other methods.

• Quantitative Bio-Science
• /
• 제36권1호
• /
• pp.23-31
• /
• 2017

Datasets from DNA microarray experiments, which are in the form of large matrices of expression levels of genes, often have missing values. However, the existing statistical methods including the principle components analysis (PCA) and Hotelling's t-test are not directly applicable for the datasets having missing values due to the fact that they assume the observed dataset is complete in general. Many methods have been proposed in previous literature to impute the missing in the observed data. Troyanskaya et al. [1] study the k-nearest neighbor (kNN) imputation, Kim et al. [2] propose the local least squares (LLS) method and Rubin [3] propose the multiple imputation (MI) for missing values. To identify differentially expressed genes, we propose a new testing procedure when the missing exists in the observed data. The proposed procedure uses the Stouffer's z-scores and combines the test results of individual imputed samples, which are dependent to each other. We numerically show that the proposed test procedure based on MI performs better than the existing test procedures based on single imputation (SI) by comparing their ROC curves. We apply the proposed method to analyzing a public microarray data.