• Proceedings of the Korean Information Science Society Conference
• /
• 2012.06c
• /
• pp.116-118
• /
• 2012

데이터의 결측치는 데이터 마이닝 알고리즘 성능에 많은 저하를 일으킨다. 따라서 본 논문에서는 멀티태스크 러닝을 이용하여 회귀 분석시에 결측치를 효율적으로 다루는 방법을 제안한다. 데이터를 데이터의 분포에 따라서 무결점 데이터와 결측 데이터를 구분하여 태스크를 나눈 후 각각의 결과를 종합하여 최적화하는 것을 목표로 한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2021.05a
• /
• pp.463-465
• /
• 2021

센서 데이터는 여러가지 원인으로 인해 데이터 결측치가 발생할 수 있으며, 결측치로 인한 데이터의 처리 방식에 따라 데이터 분석 결과가 다르게 해석될 수 있다. 이는 펫 헬스 케어 서비스에서 치명적인 문제로 연결될 수 있다. 따라서 본 논문에서는 펫 웨어러블 디바이스로부터 수집되는 다양한 센서 데이터의 결측치를 처리하기 위해 GATs(Graph Attention neTworks)와 LSTM(Long Short Term Memory)을 결합하여 활용한 데이터 결측치 처리 기법을 제안한다. 펫 웨어러블 디바이스의 센서 데이터가 서로 연관성을 가지고 있다는 점을 바탕으로 인접 노드의 Attention 수치와 Feature map을 도출한다. 이후 Prediction Layer 를 통해 결측치의 Feature 를 예측한다. 예측된 Feature 를 기반으로 Decoding 과정과 함께 결측치 보간이 이루어진다. 제안된 기법은 모델의 변형을 통해 이상치 탐지에도 활용할 수 있을 것으로 기대한다.

• The Journal of The Korea Institute of Intelligent Transport Systems
• /
• v.10 no.6
• /
• pp.63-73
• /
• 2011

Travel speed is an important parameter for measuring road traffic. UTIS(Urban Traffic Information System) was developed as a mobile detector for measuring link travel speeds in South Korea. After investigation, we founded that UTIS includes some missing data caused by the lack of probe vehicles on road segments, system failures and etc. Imputation is the practice of filling in missing data with estimated values. In this paper, we suggests a new model for imputing missing data to provide accurate link travel speeds to the public. In the field test, new model showed the travel speed measuring accuracy of 93.6%. Therefore, it can be concluded that the proposed model significantly improves travel speed measuring accuracy.

• The Korean Journal of Applied Statistics
• /
• v.20 no.3
• /
• pp.499-514
• /
• 2007

It is a common practice to use the auxiliary variables to impute missing values from item nonresponse in surveys. Sometimes there are few auxiliary variables for missing value imputation, but if spatial and time autocorrelations exist, we should use these correlations for better results. Recently, Lee et al. (2006) showed that spatial autocorrelation could be efficiently used for missing value imputation when spatial autocorrelation existed, using the data from the farm household economy data in Gangwon-do, 2002. In this paper, we present au evaluation of spatial and space-time nonresponse imputation methods when there exist spatial and time autocorrelations using the monthly data during 2000-2002 from the same data previously used by Lee et al. (2006). We show that space-time imputation method is more efficient than the other through the numerical simulations.

• Journal of Korean Society of Coastal and Ocean Engineers
• /
• v.33 no.6
• /
• pp.333-344
• /
• 2021

Technique for the long-gap filling that occur frequently in ocean monitoring data is developed. The method estimates the unknown values of the long-gap by the summation of the estimated trend and selected residual components of the given missing intervals. The method was used to impute the data of the long-term missing interval of about 1 month, such as temperature and water temperature of the Ulleungdo ocean buoy data. The imputed data showed differences depending on the monitoring parameters, but it was found that the variation pattern was appropriately reproduced. Although this method causes bias and variance errors due to trend and residual components estimation, it was found that the bias error of statistical measure estimation due to long-term missing is greatly reduced. The mean, and the 90% confidence intervals of the gap-filling model's RMS errors are 0.93 and 0.35~1.95, respectively.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2010.05a
• /
• pp.1196-1200
• /
• 2010

본 논문에서는 지점 강우의 결측치를 추정하기 위해 전통적인 통계학적 내삽기법을 이용한 역거리가중치법(IDWM), 역지수가중치법(IEWM), 상관계수가중치법(CCWM)과 패턴 인식의 일종인 인공신경망(ANN)기법 그리고 시공간적 강우분포의 측정이 가능한 레이더 자료를 이용해 결측치를 추정하여 각각의 방법을 비교하였다. 임진강 유역의 15개 지상관측소를 대상으로 교차검정(Cross validation) 분석을 실시해 본 결과, CCWM 방법과 ANN기법에 의한 RMSE가 0.46~1.79의 범위를 보였고, 보정레이더를 이용하여 결측치를 추정한 경우RMSE가 0.05~2.26의 범위를 보여 기존의 전통적 결측치 추정방법보다 실측치에 가까운 결과를 보였다. 이는 레이더자료가 지점 강우자료와는 달리 강우의 시공간적 변동성을 고려한 공간분포의 정보를 지니고 있기 때문인 것으로 판단된다.

• Journal of KIISE:Software and Applications
• /
• v.32 no.2
• /
• pp.99-107
• /
• 2005

In this paper, wc propose a now missing imputation method for minimizing loss of information linkage disequilibrium-based and haplotype-based imputation method, which estimate missing values of the data based on the specificity of Single Nucleotide Polymorphism(SNP) genotype data. Method for imputing data is needed to minimize the loss of information caused by experimental missing data. In general, missing imputation of biological data has used major allele imputation method. but this approach is not optima]. 1'his method has high error rates of missing values estimation since the characteristics of the genotype data are not considered not take into consideration the specific structure of the data. In this paper, we show the results of the comparative evaluation of our model methods and major imputation method for the estimation of missing values.

• Journal of the Korea Society of Computer and Information
• /
• v.29 no.4
• /
• pp.39-46
• /
• 2024

Many researchers make efforts to evaluate water quality using various models. Such models require a dataset without missing values, but in real world, most datasets include missing values for various reasons. Simple deletion of samples having missing value(s) could distort distribution of the underlying data and pose a significant risk of biasing the model's inference when the missing mechanism is not MCAR. In this study, to explore the most appropriate technique for handing missing values in water quality data, several imputation techniques were experimented based on existing KNN and MICE imputation with/without the generative neural network model, Autoencoder(AE) and Denoising Autoencoder(DAE). The results shows that KNN and MICE combined imputation without generative networks provides the closest estimated values to the true values. When evaluating binary classification models based on support vector machine and ensemble algorithms after applying the combined imputation technique to the observed water quality dataset with missing values, it shows better performance in terms of Accuracy, F1 score, RoC-AuC score and MCC compared to those evaluated after deleting samples having missing values.

• Journal of Korean Society of Transportation
• /
• v.27 no.4
• /
• pp.175-181
• /
• 2009

Intelligent Transportation Systems(ITS) enables road users to enhance efficiency of their trips in a variety of traffic conditions. As a significant part of ITS, information communication technology among vehicles and between vehicles and infrastructure has been being developed to upgrade current traffic data collection technology through location-based traffic surveillance systems. A wider and detailed range of traffic data can be acquired with ease by the technology. However, its performance level falls with environmental impediments such as large vehicles, buildings, harsh weather, which often bring about wireless communication failure. For imputation of vehicle trajectory data discontinued by the failure, several potential existing methods were reviewed and a new method to complement them was devised. AIMSUN API(Application Programming Interface) software was utilized to simulate vehicle trajectories data and missing vehicle trajectories data was randomly generated for the verification of the method. The method was proven to yield more accurate and reliable traffic data than the existing ones.

• Korean Journal of Remote Sensing
• /
• v.36 no.6_1
• /
• pp.1407-1419
• /
• 2020

Long time-series gridded data is crucial for the analyses of Earth environmental changes. Climate reanalysis and satellite images are now used as global-scale periodical and quantitative information for the atmosphere and land surface. This paper examines the feasibility of DCT-PLS (penalized least square regression based on discrete cosine transform) for the spatial gap filling of gridded data through the experiments for multiple variables. Because gap-free data is required for an objective comparison of original with gap-filled data, we used LDAPS (Local Data Assimilation and Prediction System) daily data and MODIS (Moderate Resolution Imaging Spectroradiometer) monthly products. In the experiments for relative humidity, wind speed, LST (land surface temperature), and NDVI (normalized difference vegetation index), we made sure that randomly generated gaps were retrieved very similar to the original data. The correlation coefficients were over 0.95 for the four variables. Because the DCT-PLS method does not require ancillary data and can refer to both spatial and temporal information with a fast computation, it can be applied to operative systems for satellite data processing.