• Communications for Statistical Applications and Methods
• /
• v.16 no.5
• /
• pp.813-820
• /
• 2009

In this paper, we review the utility-based generalization of the Shannon entropy and Kullback-Leibler information measure as the U-entropy and the U-relative entropy that was introduced by Friedman et al. (2007). Then, we derive some relations between the U-relative entropy and other information measures based on a parametric family of utility functions.

• Journal of the Korean Data and Information Science Society
• /
• v.16 no.3
• /
• pp.705-712
• /
• 2005

Entropy is the basic concept of information theory. It is well defined for random varibles with known probability density function(pdf). For given data with unknown pdf, entropy should be estimated. Usually, estimation of entropy is based on the approximations. In this paper, we consider a kernel based approximation and compare it to the cumulant approximation method for several distributions. Monte carlo simulation for various sample size is conducted.

• Proceeding of EDISON Challenge
• /
• 2017.03a
• /
• pp.75-81
• /
• 2017

Polymer collapse transition에 대한 연구가 많이 진행되어왔다. 허나 각각의 microstate에 대한 entropy나 free energy에 대한 계산을 하지는 못하였다. 최근 local nonequilibrium thermodynamics와 관련한 논문이 발표되었는데 이는 비평형 상태에서의 각각의 microstate에 대한 확률 분포를 결정하는 물리량을 발견 및 특성을 규명하여 이 중 특별한 상태가 지니는 "information" 이라는 양이 내부에너지와 엔트로피와의 상관관계가 있음을 보였다. 또한, 이러한 information theory를 이용한 Shannon entropy를 사용하여 entropy를 정의하고 free energy와 같은 물리량을 계산하였다. 따라서 이를 이용하여 information theory를 이용한 Shannon entropy와 이로 정의된 free energy를 이용하여 polymer collapse중 entropy 및 free energy를 계산하였다.

• Proceedings of the KIEE Conference
• /
• 2008.04a
• /
• pp.99-100
• /
• 2008

This paper proposes a Gaussian Mixture Model (GMM) - based speech emotion recognition methods using four feature parameters; 1) Fast Fourier Transform(FFT) spectral entropy, 2) delta FFT spectral entropy, 3) Mel-frequency Filter Bank (MFB) spectral entropy, and 4) delta MFB spectral entropy. In addition, we use four emotions in a speech database including anger, sadness, happiness, and neutrality. We perform speech emotion recognition experiments using each pre-defined emotion and gender. The experimental results show that the proposed emotion recognition using FFT spectral-based entropy and MFB spectral-based entropy performs better than existing emotion recognition based on GMM using energy, Zero Crossing Rate (ZCR), Linear Prediction Coefficient (LPC), and pitch parameters. In experimental Results, we attained a maximum recognition rate of 75.1% when we used MFB spectral entropy and delta MFB spectral entropy.

• Journal of the Korean Institute of Telematics and Electronics
• /
• v.20 no.2
• /
• pp.16-22
• /
• 1983

In this paper, we obtained PFME(probability for maximum entropy) and entropy when a conditional probability was given in a binary list order Markov Information Source. And, when steady state probability was constant, the influence of change of a conditional probability on entropy was examined, too.

• Advances in nano research
• /
• v.12 no.3
• /
• pp.263-279
• /
• 2022

In this research, the physical education training quality was investigated using the entropy model to compute variance associated with a random value (a strong tool). The entropy and undefined estimation principles are used to extract the greatest entropy of information dependent on the index system. In the study of tennis motion tracking from a dynamic viewpoint, such stages are utilized to improve the perception of the players' achievement (Lv et al. 2020). Six female tennis players served on the right side (50 cm from the T point). The initial flat serve from T point was the movement under consideration, and the entropy was utilized to weigh all indications. As a result, a multi-index measurement vector is stabilized, followed by the confidence level to determine the structural plane establishment range. As a result, the use of the unascertained measuring technique of information entropy showed an excellent approach to assessing athlete performance more accurately than traditional ways, enabling coaches and athletes to enhance their movements successfully.

• Journal of applied mathematics & informatics
• /
• v.35 no.1_2
• /
• pp.45-58
• /
• 2017

Recently, Sati and Gupta (2015) proposed two measures of uncertainty based on non-extensive entropy, called the dynamic cumulative residual Tsallis entropy (DCRTE) and the empirical cumulative Tsallis entropy. In the present paper, we extend the definition of DCRTE into the bivariate setup and study its properties in the context of reliability theory. We also define a new class of life distributions based on bivariate DCRTE.

• International Journal of Contents
• /
• v.11 no.2
• /
• pp.57-62
• /
• 2015

Relative entropy is a divergence measure between two probability density functions of a random variable. Assuming that the random variable has only two alphabets, the relative entropy becomes a cross-entropy error function that can accelerate training convergence of multi-layer perceptron neural networks. Also, the n-th order extension of cross-entropy (nCE) error function exhibits an improved performance in viewpoints of learning convergence and generalization capability. In this paper, we derive a new divergence measure between two probability density functions from the nCE error function. And the new divergence measure is compared with the relative entropy through the use of three-dimensional plots.

• Journal of the Korean Mathematical Society
• /
• v.50 no.4
• /
• pp.879-897
• /
• 2013

Shannon entropy is one of the widely used randomness measures especially for cryptographic applications. However, the conventional entropy tests are less sensitive to the inter-bit dependency in random samples. In this paper, we propose new online randomness test schemes for true random number generators (TRNGs) based on the mutual information between consecutive ${\kappa}$-bit output blocks for testing of inter-bit dependency in random samples. By estimating the block entropies of distinct lengths at the same time, it is possible to measure the mutual information, which is closely related to the amount of the statistical dependency between two consecutive data blocks. In addition, we propose a new estimation method for entropies, which accumulates intermediate values of the number of frequencies. The proposed method can estimate entropy with less samples than Maurer-Coron type entropy test can. By numerical simulations, it is shown that the new proposed scheme can be used as a reliable online entropy estimator for TRNGs used by cryptographic modules.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.35 no.9A
• /
• pp.896-905
• /
• 2010

In this paper, we present a new sensing method based on phase entropy. Entropy is a measurement which quantifies the information content contained in a signal. For the PSK modulation, the information is encoded in the phase of the transmitted signal. By focusing on phase, more information is collected during sensing, which suggests a superior performance. The sensing based on Phase entropy is not limited to PSK signal. We generalize it to PAM signal as well. It is more advantageous to detect the phase. The simulation results have confirmed the excellent performance of this novel sensing algorithm.