• Title/Summary/Keyword: data bias

Search Result 1,764, Processing Time 0.032 seconds

Review and Applications of NLL Estimation Method for Diffusion Processes (확산모형에 대한 NLL 추정법의 특성과 적용)

  • Hong, Jin-Young;Lee, Yoon-Dong
    • Communications for Statistical Applications and Methods
    • /
    • v.17 no.4
    • /
    • pp.599-609
    • /
    • 2010
  • Many of financial data are explained via diffusion models in modern financial research. Various types of estimation methods of diffusion processes were suggested by many authors. In this paper, we tested the properties of the NLL estimation method, suggested by Shoji and Ozaki (1998), of diffusion processes in the view of the bias and variance of the estimators and applied the method to estimate the model parameters for the U.S. fedral funds rate data and Korean inter-bank exchange rate data. By simulation study we showed that the NLL method provides relatively good estimators, in the meaning that the estimator has less bias than the Euler method, while keeping the variance similar level. We also provide the NLL estimates of U.S fedral funds rate data and Korean inter-bank exchange rate data.

The Study on the Oceanic Surface Wind Retrieval using TRMM Microwave Imager (TRMM TMI를 이용한 해상풍 추정에 관한 연구)

  • Kim, Young-Seup;Hong, Gi-Man
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.5 no.2
    • /
    • pp.47-53
    • /
    • 2002
  • Ocean surface wind speed was estimated using TRMM (Tropical Rainfall Measurement Mission) TMI (TRMM Microwave/Imager) data. It is used the TRMM TMI brightness temperature and National Data Buoy Center's buoy winds speed dataset near North-America to estimate by the algorithm of the ocean surface wind speed retrieval over North America. Comparing with the buoy data by D-matrix equation, the result that RMSE, BIAS, and correlation coefficient are 2.19 $ms^{-1}$, 1.10 $ms^{-1}$, and 0.81, respectively. Therefore the estimated oceanic surface wind speed by TRMM TMI brightness temperature data show that available to ocean research over upper ocean.

  • PDF

Analysis of Relationship Between Meteorological Parameters and Solar Radiation at Cheongju (청주지역의 기상요소와 일사량과의 상관관계 분석)

  • Baek, Shin Chul;Shin, Hyoung Sub;Park, Jong Hwa
    • KCID journal
    • /
    • v.19 no.1
    • /
    • pp.87-96
    • /
    • 2012
  • Information of local solar radiation is essential for many field, including water resources management, crop yield estimation, crop growth model, solar energy systems and irrigation and drainage design. Unfortunately, solar radiation measurements are not easily available due to the cost and maintenance and calibration requirements of the measuring equipment and station. Therefore, it is important to elaborate methods to estimate the solar radiation based on readily available meteorological data. In this study, two empirical equations are employed to estimate daily solar radiation using Cheongju Regional Meteorological Office data. Two scenarios are considered: (a) sunshine duration data are available for a given location, or (b) only daily cloudiness index records exist. Simple linear regression with daily sunshine duration and cloudiness index as the dependent variable accounted for 91% and 80%, respectively of the variation of solar radiation(H) at 2011. Daily global solar radiation is highly correlated with sunshine duration. In order to indicate the performance of the models, the statistical test methods of the mean bias error(MBE), root mean square error(RMSE) and correlation coefficient(r) are used. Sunshine duration and cloudiness index can be easily and reliably measured and data are widely available.

  • PDF

A Study on the Construction of Stable Clustering by Minimizing the Order Bias (순서 바이어스 최소화에 의한 안정적 클러스터링 구축에 관한 연구)

  • Lee, Gye-Seong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.6
    • /
    • pp.1571-1580
    • /
    • 1999
  • When a hierarchical structure is derived from data set for data mining and machine learning, using a conceptual clustering algorithm, one of the unsupervised learning paradigms, it is not unusual to have a different set of outcomes with respect to the order of processing data objects. To overcome this problem, the first classification process is proceeded to construct an initial partition. The partition is expected to imply the possible range in the number of final classes. We apply center sorting to the data objects in the classes of the partition for new data ordering and build a new partition using ITERATE clustering procedure. We developed an algorithm, REIT that leads to the final partition with stable and best partition score. A number of experiments were performed to show the minimization of order bias effects using the algorithm.

  • PDF

Bias-correction of Dual Polarization Radar rainfall using Convolutional Autoencoder

  • Jung, Sungho;Le, Xuan Hien;Oh, Sungryul;Kim, Jeongyup;Lee, GiHa
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2020.06a
    • /
    • pp.166-166
    • /
    • 2020
  • Recently, As the frequency of localized heavy rains increases, the use of high-resolution radar data is increasing. The produced radar rainfall has still gaps of spatial and temporal compared to gauge observation rainfall, and in many studies, various statistical techniques are performed for correct rainfall. In this study, the precipitation correction of the S-band Dual Polarization radar in use in the flood forecast was performed using the ConvAE algorithm, one of the Convolutional Neural Network. The ConvAE model was trained based on radar data sets having a 10-min temporal resolution: radar rainfall data, gauge rainfall data for 790minutes(July 2017 in Cheongju flood event). As a result of the validation of corrected radar rainfall were reduced gaps compared to gauge rainfall and the spatial correction was also performed. Therefore, it is judged that the corrected radar rainfall using ConvAE will increase the reliability of the gridded rainfall data used in various physically-based distributed hydrodynamic models.

  • PDF

Compatibility of DOAS and Conventional Point Monitoring System Through an Evaluation of Bias Structures Using Long-term Measurement Data in Seoul (장기관측자료를 이용한 DOAS와 점측정 분석시스템의 바이어스 구조에 대한 평가)

  • 김기현;김민영
    • Journal of Korean Society for Atmospheric Environment
    • /
    • v.17 no.5
    • /
    • pp.395-405
    • /
    • 2001
  • To make an assessment of the compatibility between DOAS and conventional point monitoring system (MCSAM-2: MS2), we investigated the concentrations of three criteria pollutants which include S $O_2$, N $O_2$, and $O_3$from a national monitoring station in Seoul during the periods of June 1999~August 2000. The average concentration values for the whole study period derived from hourly concentration data sets of those three species indicated that the mean differences between the two methods can be approximated as 18%. When the bias structure of two systems was evaluated through the computation of percent difference(PD) between the two such as ( $C_{DOAS}$- $C_{conventional}$ $C_{DOAS}$*100, differences between the two systems appeared to be quite systematic among different compounds. While the mode of bias peaked at 0~20% or 20~40% in terms of PD values, the cause of such positive bias mainly arised from generally enhanced concentration values of DOAS system. The structure of bias among different species was further assessed through linear regression analysis. Results of the analysis indicated that the dominant portions of differences observed from two monitoring systems can be accounted for by the systematic differences in their spanning and zeroing systems. S $O_2$(MS2)=0.6385 S $O_2$(DOAS)+2.0985($r^2$=0.7894) N $O_2$(MS2)=0.6548 N $O_2$(DOAS)+7.437($r^2$=0.7687) $O_3$(MS2)=1.0359 $O_3$(DOAS)-7.7885($r^2$=0.7944) The findings of slope values at around 0.64~0.65 from two species suggest that DOAS should respond more sensitively in upper bound concentration range. The offset values apart from zero indicate that more deliberate comparison needs to be made between these monitoring systems. However, based on the existence of strong correlations from at least 8,000 data points for each species of comparison, we were able to conclude that the compatibility of two monitoring systems is highly significant. With the improvement of calibration techniques for the DOAS system. its applicability for routine monitoring of airborne pollutant species is expected to be quite extendable.

  • PDF

A selection of optimal method for bias-correction in Global Seasonal Forecast System version 5 (GloSea5) (전지구 계절예측시스템 GloSea5의 최적 편의보정기법 선정)

  • Son, Chanyoung;Song, Junghyun;Kim, Sejin;Cho, Younghyun
    • Journal of Korea Water Resources Association
    • /
    • v.50 no.8
    • /
    • pp.551-562
    • /
    • 2017
  • In order to utilize 6-month precipitation forecasts (6 months at maximum) of Global Seasonal Forecast System version 5 (GloSea5), which is being provided by KMA (Korea Meteorological Administration) since 2014, for water resources management as well as other applications, it is needed to correct the forecast model's quantitative bias against observations. This study evaluated applicability of bias-correction skill in GloSea5 and selected an optimal method among 11 techniques that include probabilistic distribution type based, parametric, and non-parametric bias-correction to fix GloSea5's bias in precipitation forecasts. Non-parametric bias-correction provided the most similar results with observed data compared to other techniques in hindcast for the past events, yet relatively generated some discrepancies in forecast. On the contrary, parametric bias-correction produced the most reliable results in both hindcast and forecast periods. The results of this study are expected to be applicable to various applications using seasonal forecast model such as water resources operation and management, hydropower, agriculture, etc.

Development of robust Calibration for Determination Sweetness of Fuji Apple fruit using Near Infrared Reflectance Spectroscopy

  • Sohn, Mi-Ryeong;Kwon, Young-Kill;Cho, Rae-Kwang
    • Near Infrared Analysis
    • /
    • v.2 no.1
    • /
    • pp.55-58
    • /
    • 2001
  • The object of this work was to investigate the influence of growing district and harvest year on calibration for sweetness (Brix) determination of Fuji apple fruit using near infrared (NIR) reflectance spectroscopy, and to develop the robust calibration across these variation. The calibration models was based on wavelength range of 1100∼2500 nm using a stepwise multiple linear regression. A calibration model by sample set of one growing district was not transferable to other growing districts. The combined calibration (data of three growing districts) predicted reasonable well against a population set drawn from all growing districts (SEP=0.69, Bias=0.075). A calibration model by sample set of one harvest year was not also transferable to other harvest years. The combined calibration (data of three harvest years) predicted well against a population set drawn from all harvest years (SEP=0.53, Bias=0.004).

Sensitivity analysis in Bayesian nonignorable selection model for binary responses

  • Choi, Seong Mi;Kim, Dal Ho
    • Journal of the Korean Data and Information Science Society
    • /
    • v.25 no.1
    • /
    • pp.187-194
    • /
    • 2014
  • We consider a Bayesian nonignorable selection model to accommodate the selection bias. Markov chain Monte Carlo methods is known to be very useful to fit the nonignorable selection model. However, sensitivity to prior assumptions on parameters for selection mechanism is a potential problem. To quantify the sensitivity to prior assumption, the deviance information criterion and the conditional predictive ordinate are used to compare the goodness-of-fit under two different prior specifications. It turns out that the 'MLE' prior gives better fit than the 'uniform' prior in viewpoints of goodness-of-fit measures.

An Analysis on Efficiency for the Environmental Friendly Agricultural Product of Strawberry in GyeongBuk Province (경북지역 친환경딸기 농가의 인증유형에 따른 효율성 분석)

  • Lee, Sang-Ho;Song, Kyung-Hwan
    • Korean Journal of Organic Agriculture
    • /
    • v.21 no.4
    • /
    • pp.487-500
    • /
    • 2013
  • The purpose of this study is to estimate efficiency of environmental-friendly agricultural product by using Data Envelopment Analysis. A proposed method employs a bootstrapping approach to generating efficiency estimates through Monte Carlo simulation resampling process. The technical efficiency, pure technical efficiency, and scale efficiency measure of strawberry by pesticide-free certification is 0.967, 0.995, 0.968 respectively. However those of bias-corrected estimates are 0.918, 0.983, 0.934. We know that the DEA estimator is an upward biased estimator. In technical efficiency, average lower and upper confidence bounds of 0.807 and 0.960. According to these results, the DEA bootstrapping model used here provides bias-corrected and confidence intervals for the point estimates, it is more preferable.