• Communications for Statistical Applications and Methods
• /
• v.27 no.4
• /
• pp.469-486
• /
• 2020

Entropy is an important term in statistical mechanics that was originally defined in the second law of thermodynamics. In this paper, we consider the maximum likelihood estimation (MLE), maximum product spacings estimation (MPSE) and Bayesian estimation of the entropy of an inverse Weibull distribution (InW) under a generalized type I progressive hybrid censoring scheme (GePH). The MLE and MPSE of the entropy cannot be obtained in closed form; therefore, we propose using the Newton-Raphson algorithm to solve it. Further, the Bayesian estimators for the entropy of InW based on squared error loss function (SqL), precautionary loss function (PrL), general entropy loss function (GeL) and linex loss function (LiL) are derived. In addition, we derive the Lindley's approximate method (LiA) of the Bayesian estimates. Monte Carlo simulations are conducted to compare the results among MLE, MPSE, and Bayesian estimators. A real data set based on the GePH is also analyzed for illustrative purposes.

• Genomics & Informatics
• /
• v.20 no.2
• /
• pp.17.1-17.11
• /
• 2022

Genetic associations have been quantified using a number of statistical measures. Entropy-based mutual information may be one of the more direct ways of estimating the association, in the sense that it does not depend on the parametrization. For this purpose, both the entropy and conditional entropy of the phenotype distribution should be obtained. Quantitative traits, however, do not usually allow an exact evaluation of entropy. The estimation of entropy needs a probability density function, which can be approximated by kernel density estimation. We have investigated the proper sequence of procedures for combining the kernel density estimation and entropy estimation with a probability density function in order to calculate mutual information. Genotypes and their interactions were constructed to set the conditions for conditional entropy. Extensive simulation data created using three types of generating functions were analyzed using two different kernels as well as two types of multifactor dimensionality reduction and another probability density approximation method called m-spacing. The statistical power in terms of correct detection rates was compared. Using kernels was found to be most useful when the trait distributions were more complex than simple normal or gamma distributions. A full-scale genomic dataset was explored to identify associations using the 2-h oral glucose tolerance test results and γ-glutamyl transpeptidase levels as phenotypes. Clearly distinguishable single-nucleotide polymorphisms (SNPs) and interacting SNP pairs associated with these phenotypes were found and listed with empirical p-values.

• Communications for Statistical Applications and Methods
• /
• v.18 no.4
• /
• pp.457-465
• /
• 2011

Sample entropy (Vasicek, 1976) has poor performance, and several nonparametric entropy estimators have been proposed as alternatives. In this paper, we consider a piecewise uniform density function based on quantiles, which enables us to evaluate entropy in each interval, and study the poor performance of the sample entropy in terms of the poor estimation of lower and upper quantiles. Then we propose some improved entropy estimators by simply modifying the quantile estimators, and compare their performances with some existing estimators.

• Communications for Statistical Applications and Methods
• /
• v.12 no.1
• /
• pp.125-137
• /
• 2005

Many discriminant analysis models for binary data have been used in real applications, but none of the classification models dominates in all varying circumstances(Asparoukhov & Krzanowski(2001)). Lee and Hwang (2003) proposed a new classification model by using multinomial distribution with the maximum entropy estimation method. The model showed some promising results in case of small number of variables, but its performance was not satisfactory for large number of variables. This paper explores to use the iterative cross entropy minimization estimation method in replace of the maximum entropy estimation. Simulation experiments show that this method can compete with other well known existing classification models.

• Communications for Statistical Applications and Methods
• /
• v.13 no.3
• /
• pp.657-667
• /
• 2006

The concept of entropy, introduced in communication theory by Shannon (1948) as a measure of uncertainty, is of prime interest in information-theoretic statistics. This paper considers the minimum variance unbiased estimation for the maximum entropy of the transformed inverse Gaussian random variable by $Y=X^{-1/2}$. The properties of the derived UMVU estimator is investigated.

• Journal of the Korean Data and Information Science Society
• /
• v.28 no.3
• /
• pp.659-668
• /
• 2017

The inverse Weibull distribution (IWD) can be readily applied to a wide range of situations including applications in medicines, reliability and ecology. It is generally known that the lifetimes of test items may not be recorded exactly. In this paper, therefore, we consider the maximum likelihood estimation (MLE) and Bayes estimation of the entropy of a IWD under generalized progressive hybrid censoring (GPHC) scheme. It is observed that the MLE of the entropy cannot be obtained in closed form, so we have to solve two non-linear equations simultaneously. Further, the Bayes estimators for the entropy of IWD based on squared error loss function (SELF), precautionary loss function (PLF), and linex loss function (LLF) are derived. Since the Bayes estimators cannot be obtained in closed form, we derive the Bayes estimates by revoking the Tierney and Kadane approximate method. We carried out Monte Carlo simulations to compare the classical and Bayes estimators. In addition, two real data sets based on GPHC scheme have been also analysed for illustrative purposes.

• Water Engineering Research
• /
• v.4 no.3
• /
• pp.155-173
• /
• 2003

Parameters and quantiles of the 2-parameter generalized Pareto distribution were estimated using the methods of regular moments, modified moments, probability weighted moments, linear moments, maximum likelihood, and entropy for Monte Carlo-generated samples. The performance of these seven estimators was statistically compared, with the objective of identifying the most robust estimator. It was found that in general the methods of probability-weighted moments and L-moments performed better than the methods of maximum likelihood estimation, moments and entropy, especially for smaller values of the coefficient of variation and probability of exceedance.

• Journal of the Korean Data and Information Science Society
• /
• v.16 no.3
• /
• pp.705-712
• /
• 2005

Entropy is the basic concept of information theory. It is well defined for random varibles with known probability density function(pdf). For given data with unknown pdf, entropy should be estimated. Usually, estimation of entropy is based on the approximations. In this paper, we consider a kernel based approximation and compare it to the cumulant approximation method for several distributions. Monte carlo simulation for various sample size is conducted.

• Atmosphere
• /
• v.25 no.3
• /
• pp.511-520
• /
• 2015

Existing studies on radar rainfall uncertainties were performed to reduce the uncertainty for each stage by using bias correction during the quantitative radar rainfall estimation process. However, the studies do not provide quantitative comparison with the uncertainties for all stages. Consequently, this study proposes a suitable approach that can quantify the uncertainties at each stage of the quantitative radar rainfall estimation process. First, the new approach can present initial and final uncertainties, increasing or decreasing the uncertainty, and the uncertainty percentage at each stage. Furthermore, Maximum Entropy (ME) was applied to quantify the uncertainty in the entire process. Second, for the uncertainty quantification of radar rainfall estimation at each stage, this study used two quality control algorithms, two rainfall estimation relations, and two bias correction techniques as post-processing and progressed through all stages of the radar rainfall estimation. For the proposed approach, the final uncertainty (ME = 3.81) from the ME of the bias correction stage was the smallest while the uncertainty of the rainfall estimation stage was higher because of the use of an unsuitable relation. Additionally, the ME of the quality control was at 4.28 (112.34%), while that of the rainfall estimation was at 4.53 (118.90%), and that of the bias correction at 3.81 (100%). However, this study also determined that selecting the appropriate method for each stage would gradually reduce the uncertainty at each stage. Finally, the uncertainty due to natural variability was 93.70% of the final uncertainty. Thus, the results indicate that this new approach can contribute significantly to the field of uncertainty estimation and help with estimating more accurate radar rainfall.

• ETRI Journal
• /
• v.42 no.3
• /
• pp.311-323
• /
• 2020

Encrypted traffic classification plays a vital role in cybersecurity as network traffic encryption becomes prevalent. First, we briefly introduce three traffic encryption mechanisms: IPsec, SSL/TLS, and SRTP. After evaluating the performances of support vector machine, random forest, naïve Bayes, and logistic regression for traffic classification, we propose the combined approach of entropy estimation and artificial neural networks. First, network traffic is classified as encrypted or plaintext with entropy estimation. Encrypted traffic is then further classified using neural networks. We propose using traffic packet's sizes, packet's inter-arrival time, and direction as the neural network's input. Our combined approach was evaluated with the dataset obtained from the Canadian Institute for Cybersecurity. Results show an improved precision (from 1 to 7 percentage points), and some application classification metrics improved nearly by 30 percentage points.