• Title, Summary, Keyword: Kullback-Leibler distance

Search Result 21, Processing Time 0.029 seconds

Video Content Indexing using Kullback-Leibler Distance

  • Kim, Sang-Hyun
    • International Journal of Contents
    • /
    • v.5 no.4
    • /
    • pp.51-54
    • /
    • 2009
  • In huge video databases, the effective video content indexing method is required. While manual indexing is the most effective approach to this goal, it is slow and expensive. Thus automatic indexing is desirable and recently various indexing tools for video databases have been developed. For efficient video content indexing, the similarity measure is an important factor. This paper presents new similarity measures between frames and proposes a new algorithm to index video content using Kullback-Leibler distance defined between two histograms. Experimental results show that the proposed algorithm using Kullback-Leibler distance gives remarkable high accuracy ratios compared with several conventional algorithms to index video content.

SOME INEQUALITIES FOR THE $CSISZ{\acute{A}}R\;{\Phi}-DIVERGENCE$

  • Dragomir, S.S.
    • Journal of the Korean Society for Industrial and Applied Mathematics
    • /
    • v.7 no.1
    • /
    • pp.63-77
    • /
    • 2003
  • Some inequalities for the $Csisz{\acute{a}}r\;{\Phi}-divergence$ and applications for the Kullback-Leibler, $R{\acute{e}}nyi$, Hellinger and Bhattacharyya distances in Information Theory are given.

  • PDF

Performance Improvement of Ensemble Speciated Neural Networks using Kullback-Leibler Entropy (Kullback-Leibler 엔트로피를 이용한 종분화 신경망 결합의 성능향상)

  • Kim, Kyung-Joong;Cho, Sung-Bae
    • The Transactions of the Korean Institute of Electrical Engineers D
    • /
    • v.51 no.4
    • /
    • pp.152-159
    • /
    • 2002
  • Fitness sharing that shares fitness if calculated distance between individuals is smaller than sharing radius is one of the representative speciation methods and can complement evolutionary algorithm which converges one solution. Recently, there are many researches on designing neural network architecture using evolutionary algorithm but most of them use only the fittest solution in the last generation. In this paper, we elaborate generating diverse neural networks using fitness sharing and combing them to compute outputs then, propose calculating distance between individuals using modified Kullback-Leibler entropy for improvement of fitness sharing performance. In the experiment of Australian credit card assessment, breast cancer, and diabetes in UCI database, proposed method performs better than not only simple average output or Pearson Correlation but also previous published methods.

Class Determination Based on Kullback-Leibler Distance in Heart Sound Classification

  • Chung, Yong-Joo;Kwak, Sung-Woo
    • The Journal of the Acoustical Society of Korea
    • /
    • v.27 no.2E
    • /
    • pp.57-63
    • /
    • 2008
  • Stethoscopic auscultation is still one of the primary tools for the diagnosis of heart diseases due to its easy accessibility and relatively low cost. It is, however, a difficult skill to acquire. Many research efforts have been done on the automatic classification of heart sound signals to support clinicians in heart sound diagnosis. Recently, hidden Markov models (HMMs) have been used quite successfully in the automatic classification of the heart sound signal. However, in the classification using HMMs, there are so many heart sound signal types that it is not reasonable to assign a new class to each of them. In this paper, rather than constructing an HMM for each signal type, we propose to build an HMM for a set of acoustically-similar signal types. To define the classes, we use the KL (Kullback-Leibler) distance between different signal types to determine if they should belong to the same class. From the classification experiments on the heart sound data consisting of 25 different types of signals, the proposed method proved to be quite efficient in determining the optimal set of classes. Also we found that the class determination approach produced better results than the heuristic class assignment method.

CONDITIONAL LARGE DEVIATIONS FOR 1-LATTICE DISTRIBUTIONS

  • Kim, Gie-Whan
    • The Pure and Applied Mathematics
    • /
    • v.4 no.1
    • /
    • pp.97-104
    • /
    • 1997
  • The large deviations theorem of Cramer is extended to conditional probabilities in the following sense. Consider a random sample of pairs of random vectors and the sample means of each of the pairs. The probability that the first falls outside a certain convex set given that the second is fixed is shown to decrease with the sample size at an exponential rate which depends on the Kullback-Leibler distance between two distributions in an associated exponential familiy of distributions. Examples are given which include a method of computing the Bahadur exact slope for tests of certain composite hypotheses in exponential families.

  • PDF

Generalized Kullback-Leibler information and its extensions to censored and discrete cases

  • Park, Sangun
    • Journal of the Korean Data and Information Science Society
    • /
    • v.23 no.6
    • /
    • pp.1223-1229
    • /
    • 2012
  • In this paper, we propose a generalized Kullback-Leibler (KL) information for measuring the distance between two distribution functions where the extension to the censored case is immediate. The generalized KL information has the nonnegativity and characterization properties, and its censored version has the additional property of monotonic increase. We also extend the discussion to the discrete case and propose a generalized censored measure which is comparable to Pearson's chi-square statistic.

Speaker Verification Using SVM Kernel with GMM-Supervector Based on the Mahalanobis Distance (Mahalanobis 거리측정 방법 기반의 GMM-Supervector SVM 커널을 이용한 화자인증 방법)

  • Kim, Hyoung-Gook;Shin, Dong
    • The Journal of the Acoustical Society of Korea
    • /
    • v.29 no.3
    • /
    • pp.216-221
    • /
    • 2010
  • In this paper, we propose speaker verification method using Support Vector Machine (SVM) kernel with Gaussian Mixture Model (GMM)-supervector based on the Mahalanobis distance. The proposed GMM-supervector SVM kernel method is combined GMM with SVM. The GMM-supervectors are generated by GMM parameters of speaker and other speaker utterances. A speaker verification threshold of GMM-supervectors is decided by SVM kernel based on Mahalanobis distance to improve speaker verification accuracy. The experimental results for text-independent speaker verification using 20 speakers demonstrates the performance of the proposed method compared to GMM, SVM, GMM-supervector SVM kernel based on Kullback-Leibler (KL) divergence, and GMM-supervector SVM kernel based on Bhattacharyya distance.

Direct Divergence Approximation between Probability Distributions and Its Applications in Machine Learning

  • Sugiyama, Masashi;Liu, Song;du Plessis, Marthinus Christoffel;Yamanaka, Masao;Yamada, Makoto;Suzuki, Taiji;Kanamori, Takafumi
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.2
    • /
    • pp.99-111
    • /
    • 2013
  • Approximating a divergence between two probability distributions from their samples is a fundamental challenge in statistics, information theory, and machine learning. A divergence approximator can be used for various purposes, such as two-sample homogeneity testing, change-point detection, and class-balance estimation. Furthermore, an approximator of a divergence between the joint distribution and the product of marginals can be used for independence testing, which has a wide range of applications, including feature selection and extraction, clustering, object matching, independent component analysis, and causal direction estimation. In this paper, we review recent advances in divergence approximation. Our emphasis is that directly approximating the divergence without estimating probability distributions is more sensible than a naive two-step approach of first estimating probability distributions and then approximating the divergence. Furthermore, despite the overwhelming popularity of the Kullback-Leibler divergence as a divergence measure, we argue that alternatives such as the Pearson divergence, the relative Pearson divergence, and the $L^2$-distance are more useful in practice because of their computationally efficient approximability, high numerical stability, and superior robustness against outliers.

The Study on the Verification of Speaker Change using GMM-UBM based KL distance (GMM-UBM 기반 KL 거리를 활용한 화자변화 검증에 대한 연구)

  • Cho, Joon-Beom;Lee, Ji-eun;Lee, Kyong-Rok
    • Journal of Convergence Society for SMB
    • /
    • v.6 no.4
    • /
    • pp.71-77
    • /
    • 2016
  • In this paper, we proposed a verification of speaker change utilizing the KL distance based on GMM-UBM to improve the performance of conventional BIC based Speaker Change Detection(SCD). We have verified Conventional BIC-based SCD using KL-distance based SCD which is robust against difference of information volume than BIC-based SCD. And we have applied GMM-UBM to compensate asymmetric information volume. Conventional BIC-based SCD was composed of two steps. Step 1, to detect the Speaker Change Candidate Point(SCCP). SCCP is positive local maximum point of dissimilarity d. Step 2, to determine the Speaker Change Point(SCP). If ${\Delta}BIC$ of SCCP is positive, it decides to SCP. We examined verification of SCP using GMM-UBM based KL distance D. If the value of D on each SCP is higher than threshold, we accepted that point to the final SCP. In the experimental condition MDR(Missed Detection Rate) is 0, FAR(False Alarm Rate) when the threshold value of 0.028 has been improved to 60.7%.

The Study on Speaker Change Verification Using SNR based weighted KL distance (SNR 기반 가중 KL 거리를 활용한 화자 변화 검증에 관한 연구)

  • Cho, Joon-Beom;Lee, Ji-eun;Lee, Kyong-Rok
    • Journal of Convergence for Information Technology
    • /
    • v.7 no.6
    • /
    • pp.159-166
    • /
    • 2017
  • In this paper, we have experimented to improve the verification performance of speaker change detection on broadcast news. It is to enhance the input noisy speech and to apply the KL distance $D_s$ using the SNR-based weighting function $w_m$. The basic experimental system is the verification system of speaker change using GMM-UBM based KL distance D(Experiment 0). Experiment 1 applies the input noisy speech enhancement using MMSE Log-STSA. Experiment 2 applies the new KL distance $D_s$ to the system of Experiment 1. Experiments were conducted under the condition of 0% MDR in order to prevent missing information of speaker change. The FAR of Experiment 0 was 71.5%. The FAR of Experiment 1 was 67.3%, which was 4.2% higher than that of Experiment 0. The FAR of experiment 2 was 60.7%, which was 10.8% higher than that of experiment 0.