• Journal of Korea Multimedia Society
• /
• v.25 no.2
• /
• pp.257-268
• /
• 2022

When there is a missing value in the raw data, if ignore the missing values and proceed with the analysis, the accuracy decrease due to the decrease in the number of sample. The method of imputation and analyzing patterns and significant values can compensate for the problem of lower analysis quality and analysis accuracy as a result of bias rather than simply removing missing values. In this study, we proposed to study irregular data patterns and missing processing methods of data using machine learning techniques for the study of correction of missing values. we would like to propose a plan to replace the missing with data from a similar past point in time by finding the situation at the time when the missing data occurred. Unlike previous studies, data correction techniques present new algorithms using DNN and KNN-MLE techniques. As a result of the performance evaluation, the ANAE measurement value compared to the existing missing section correction algorithm confirmed a performance improvement of about 0.041 to 0.321.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.19 no.5
• /
• pp.1027-1034
• /
• 2024

This study aims to address the issues of outliers and missing values in AI-based smart farming to improve data quality and enhance the accuracy of agricultural predictive activities. By utilizing real data provided by the Rural Development Administration (RDA) and the Korea Agency of Education, Promotion, and Information Service in Food, Agriculture, Forestry, and Fisheries (EPIS), outlier detection and missing value imputation techniques were applied to collect and manage high-quality data. For successful smart farm operations, an IoT-based AI automatic growth measurement model is essential, and achieving a high data quality index through stable data preprocessing is crucial. In this study, various methods for correcting outliers and imputing missing values in growth data were applied, and the proposed preprocessing strategies were validated using machine learning performance evaluation indices. The results showed significant improvements in model performance, with high predictive accuracy observed in key evaluation metrics such as ROC and AUC.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.53 no.3
• /
• pp.99-106
• /
• 2016

In this paper, we will discuss about correction method of missing data on noisy observation matrix and uncertainty analysis for the potential noise. In situations without missing data in an observation matrix, this solution is known to be accurately induced by SVD (Singular Value Decomposition). However, usually the several entries of observation matrix have not been observed and other entries have been perturbed by the influence of noise. In this case, it is difficult to find the solution as well as cause the 3D reconstruction error. Therefore, in order to minimize the 3D reconstruction error, above all things, it is necessary to correct reliably the missing data under noise distribution and to give a quantitative evaluation for the corrected results. This paper focuses on a method for correcting missing data using geometrical properties between 2D projected object and 3D reconstructed shape and for estimating a noise level of the observation matrix using ranks of SVD in order to quantitatively evaluate the performance of the correction algorithm.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.21 no.1
• /
• pp.181-187
• /
• 2021

Recently, various air pollution factors have been measured and analyzed to reduce damages caused by it. In this process, many missing values occur due to various causes. To compensate for this, basically a vast amount of training data is required. This paper proposes a statistical techniques that effectively compensates for missing values generated in the process of measuring ozone, carbon dioxide, and ultra-fine dust using a small amount of learning data. The proposed algorithm first extracts a group of meteorological data that is expected to have positive effects on the correction of missing values through statistical information analysis such as the correlation between meteorological data and air pollution level factors, p-value, etc. It is a technique that efficiently and effectively compensates for missing values by analyzing them. In order to confirm the performance of the proposed algorithm, we analyze its characteristics through various experiments and compare the performance of the well-known representative algorithms with ours.

• The Journal of the Korea Contents Association
• /
• v.21 no.1
• /
• pp.277-283
• /
• 2021

Traffic data by vehicle classification are important data used as basic data in various fields such as road and traffic design. Traffic data is collected through permanent and temporary surveys and is provided as an annual average daily traffic (AATD) in the statistical yearbook of road traffic. permanent surveys are collected through traffic collection equipment (AVC), and the AVC consists of a loop sensor that detects traffic volume and a piezo sensor that detects the number of axes. Due to the nature of the buried type of traffic collection equipment, missing data is generated due to failure of detection equipment. In the existing method, it is corrected through historical data and the trend of traffic around the point. However, this method has a disadvantage in that it does not reflect temporal and spatial characteristics and that the existing data used for correction may also be a correction value. In this study, we proposed a method to correct the missing traffic volume by calculating the axis correction coefficient through the accumulated number of axes acquired by using a piezo sensor that can detect the axis of the vehicle. This has the advantage of being able to reflect temporal and spatial characteristics, which are the limitations of the existing methods, and as a result of comparative evaluation, the error rate was derived lower than that of the existing methods. The traffic volume correction system using axis count is judged as a correction method applicable to the field system with a simple algorithm.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.35 no.1
• /
• pp.47-56
• /
• 2012

Service Parts Management is defined as a supply management associated with service parts from the part suppliers to the final customer. A series of process to improve the customer service level by forecasting the demand and to minimize cost by maintaining the inventory level is included. Uniqueness such as missing value correction, the data pattern analysis and planned order system is designed and implemented. Main feature of order management system is to calculate order amount and order time based on selection of optimal forecasting algorithm.

• Journal of The Korean Society of Agricultural Engineers
• /
• v.63 no.5
• /
• pp.83-93
• /
• 2021

Agricultural reservoirs are important facilities for storing or managing water for the purpose of securing agricultural water, creating and expanding agricultural production bases, and using them to increase agricultural production. In particular, the Korea Rural Community Corporation (KRC) manages agricultural reservoirs scattered across the country, and officially recognizes and distributes hydrological data to increase their public utilization and aims to improve the value of water resources. Data on the water level of agricultural reservoirs are important. However, errors such as missing values and outliners limit utilization of the data in various fields of research and industry. Therefore, water quality data measures should be devised to increase reliability. this study categorized different error types and looked at automatic correction methods to enhance the reliability of the vast hydrological data. In addition, the water level data corrected from errors were compared to the reference hydrologic data through expert judgment in accordance with the quality control procedure, and the most appropriate measures were verified. As KRC manages more agricultural reservoirs than any other institution, the proposed method of efficient and automatic water level data correction in this study is expected to increase the availability and reliability of the hydrological data.

• The Journal of The Korea Institute of Intelligent Transport Systems
• /
• v.20 no.6
• /
• pp.1-13
• /
• 2021

In this study, a time series analysis technique was applied to calibrate and predict traffic data for various purposes, such as planning, design, maintenance, and research. Existing algorithms have limitations in application to data such as traffic data because they show strong periodicity and seasonality or irregular data. To overcome and supplement these limitations, we applied the SARIMA model, an analytical technique that combines the autocorrelation model, the Seasonal Auto Regressive(SAR), and the seasonal Moving Average(SMA). According to the analysis, traffic volume prediction using the SARIMA(4,1,3)(4,0,3) 12 model, which is the optimal parameter combination, showed excellent performance of 85% on average. In addition to traffic data, this study is considered to be of great value in that it can contribute significantly to traffic correction and forecast improvement in the event of missing traffic data, and is also applicable to a variety of time series data recently collected.

• Journal of Positioning, Navigation, and Timing
• /
• v.7 no.4
• /
• pp.295-305
• /
• 2018

Network RTK is a highly practical technology that can provide high positioning accuracy at levels between cm~dm regardless of user location in the network by extending the available range of RTK using reference station network. In particular, unlike other carrier-based positioning techniques such as PPP, users are able to acquire high-accuracy positions within a short initialization time of a few or tens of seconds, which increases its value as a future navigation system. However, corrections must be continuously received to maintain a high level of positioning accuracy, and when a time delay of more than 30 seconds occurs, the accuracy may be reduced to the code-based positioning level of meters. In case of SSR, which is currently in the process of standardization for PPP service, the corrections by each error source are transmitted in different transmission intervals, and the rate of change of each correction is transmitted together to compensate the time delay. Using these features of SSR correction is expected to reduce the performance degradation even if users do not receive the network RTK corrections for more than 30 seconds. In this paper, the simulation data were generated from 5 domestic reference stations in Gunwi, Yeongdoek, Daegu, Gimcheon, and Yecheon, and the network RTK and SSR corrections were generated for the corresponding data and applied to the simulation data from Cheongsong reference station, assumed as the user. As a result of the experiment assuming 30 seconds of missing data, the positioning performance compensating for time delay by SSR was analyzed to be horizontal RMS (about 5 cm) and vertical RMS (about 8 cm), and the 95% error was 8.7 cm horizontal and 1cm vertical. This is a significant amount when compared to the horizontal and vertical RMS of 0.3 cm and 0.6 cm, respectively, for Network RTK without time delay for the same data, but is considerably smaller compared to the 0.5 ~ 1 m accuracy level of DGPS or SBAS. Therefore, maintaining Network RTK mode using SSR rather than switching to code-based DGPS or SBAS mode due to failure to receive the network RTK corrections for 30 seconds is considered to be favorable in terms of maintaining position accuracy and recovering performance by quickly resolving the integer ambiguity when the communication channel is recovered.

• Journal of Korean Society of Transportation
• /
• v.35 no.1
• /
• pp.63-78
• /
• 2017

The purpose of this study is to determine the optimal aggregation interval to increase the reliability when estimating representative value of individual vehicle travel time collected by DSRC equipment in interrupted traffic flow section in National Highway. For this, we use the bimodal asymmetric distribution data, which is the distribution of the most representative individual vehicle travel time collected in the interrupted traffic flow section, and estimate the MSE(Mean Square Error) according to the variation of the aggregation interval of individual vehicle travel time, and determine the optimal aggregation interval. The estimation equation for the MSE estimation utilizes the maximum estimation error equation of t-distribution that can be used in asymmetric distribution. For the analysis of optimal aggregation interval size, the aggregation interval size of individual vehicle travel time was only 3 minutes or more apart from the aggregation interval size of 1-2 minutes in which the collection of data was normally lost due to the signal stop in the interrupted traffic flow section. The aggregation interval that causes the missing part in the data collection causes another error in the missing data correction process and is excluded. As a result, the optimal aggregation interval for the minimum MSE was 3~5 minutes. Considering both the efficiency of the system operation and the improvement of the reliability of calculation of the travel time, it is effective to operate the basic aggregation interval as 5 minutes as usual and to reduce the aggregation interval to 3 minutes in case of congestion.