Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Error Estimation Based on the Bhattacharyya Distance for Classifying Multimodal Data

Multimodal 데이터에 대한 분류 에러 예측 기법

  • 최의선 (연세대학교 전기·전자공학과) ;
  • 김재희 (연세대학교 전기·전자공학과) ;
  • 이철희 (연세대학교 전기·전자공학과)
  • Published : 2002.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an error estimation method based on the Bhattacharyya distance for multimodal data. First, we try to find the empirical relationship between the classification error and the Bhattacharyya distance. Then, we investigate the possibility to derive the error estimation equation based on the Bhattacharyya distance for multimodal data. We assume that the distribution of multimodal data can be approximated as a mixture of several Gaussian distributions. Experimental results with remotely sensed data showed that there exist strong relationships between the Bhattacharyya distance and the classification error and that it is possible to predict the classification error using the Bhattacharyya distance for multimodal data.

본 논문에서는 multimodal 특성을 갖는 데이터에 대하여 패턴 분류 시 Bhattacharyya distance에 기반한 에러 예측 기법을 제안한다. 제안한 방법은 multimodal 데이터에 대하여 분류 에러와 Bhattacharyya distance를 각각 실험적으로 구하고 이 둘 사이의 관계를 유추하여 에러의 예측 가능성을 조사한다. 본 논문에서는 분류 에러 및 Bhattacharyya distance를 구하기 위하여 multimodal 데이터의 확률 밀도 함수를 정규 분포 특성을 갖는 부클래스들의 조합으로 추정한다. 원격 탐사 데이터를 이용하여 실험한 결과, multimodal 데이터의 분류 에러와 Bhattacharyya distance 사이에 밀접한 관련이 있음이 확인되었으며, Bhattacharyya distance를 이용한 에러 예측 가능성을 보여주었다.

Keywords

References

  1. K. Fukunaga, Introduction to Statistical Pattern Recognition, New York : Academic Press, 1972
  2. K. Fukunaga and T. F. Krile, 'Calculation of Bayes' recognition error for two multivariate Gaussian distributions,' IEEE Trans. Comp., vol. 18, no. 3, pp. 220-229,1969 https://doi.org/10.1109/T-C.1969.222635
  3. D. Hand, 'Recent advances in error rate estimation,' Pattern Recognition Lett., vol. 4, pp. 335-346, 1968 https://doi.org/10.1016/0167-8655(86)90054-1
  4. K. Fukunaga and D. Kessell, 'Estimation of classification error,' IEEE Trans. Comp., vol. 20, no. 12, pp. 1521-1527, 1971 https://doi.org/10.1109/T-C.1971.223165
  5. K. Fukunaga and D. Hummels, 'Leave-one-out procedures for nonparametric error estimates,' IEEE Trans. Pattern Anal. & Machine Intelli., vol. 11, no. 4, pp. 421-423, 1989 https://doi.org/10.1109/34.19039
  6. L. Buturovic, 'Towards Bayes-optimal linear dimension reduction,' IEEE Trans. Pattern Anal. & Machine Intelli., vol. 16, pp. 420-424, 1994 https://doi.org/10.1109/34.277596
  7. R. P. Heydorn, 'An upper bound estimate on classification error,' IEEE Trans. Inform. Theory, vol. 14, pp. 783-784, 1968 https://doi.org/10.1109/TIT.1968.1054198
  8. G. Lugosi and M. Pawlak, 'On the posterior-probability estimate of the error rate of nonparametric classification rules,' IEEE Trans. Inform. Theory, vol. 40, no. 2, 1994 https://doi.org/10.1109/18.312167
  9. R. Lotlikar and R. Kothari, 'Adaptive linear dimensionality reduction for classification,' Pattern Recognition, vol. 33, pp. 185-194, 2000 https://doi.org/10.1016/S0031-3203(99)00053-9
  10. K. Tumer and J. Ghosh, 'Estimating the Bayes error rate through classifier combining,' ICPR'96, vol. 2, pp. 695-699, 1996 https://doi.org/10.1109/ICPR.1996.546912
  11. K. Tumer and J. Ghosh, 'Analysis of decision boundaries in linearly combined neural classi-fiers,' Pattern Recognition, vol. 29, no. 2, pp. 341-348, 1996 https://doi.org/10.1016/0031-3203(95)00085-2
  12. S. Raudys and V. Pikelis, 'On dimensionality, sample size, classification error, and complexity of classification algorithm in pattern recognition, ' IEEE Trans. Pattern Anal. & Machine Intelli., vol. 2, no. 3, 1980
  13. H. Schulerud, 'Bias of error in linear discriminant analysis caused by feature selec-tion and sample size,' ICPR 2000, vol. 2, pp. 372-377, 2000 https://doi.org/10.1109/ICPR.2000.906090
  14. L. Devroye, 'Any discrimination rule can have an arbitrarily bad probability of error for finite sample size,' IEEE Trans. Pattern Anal. & Machine Intelli., vol. 4, pp. 154 -157, 1982 https://doi.org/10.1109/TPAMI.1982.4767222
  15. A. Antos, L. Devroye and L. Gyorfi, 'Lower bounds for Bayes error estimation,' IEEE Trans. Pattern Anal. & Machine Intelli., vol. 21, no. 7, pp. 643-645, 1999 https://doi.org/10.1109/34.777375
  16. C. Lee and E. Choi, 'Bayes error evaluation of the Gaussian ML classifier,' IEEE Trans. Geos. Remote Sensing, vol. 38, no. 3, pp. 1471-1475, 2000 https://doi.org/10.1109/36.843045
  17. T. Kaliath, 'The divergence and Bhattacharyya distance measures in signal selection,' IEEE Trans. Commun. Technol., vol. 15, pp. 52-60, 1967 https://doi.org/10.1109/TCOM.1967.1089532
  18. J. A, Richards, Remote Sensing Digital Image Analysis, Springer-Verlag, 1993
  19. R. O. Duda and P. E. Hart, Pattern Classifi-cation and Scene Analysis, John Wiley and Sons, New York, NY, 1973
  20. T. Taxt, N. L. Hjort and L. Eikvil, 'Statistical classification using a linear mixture of two multinormal probability densities,' Pattern Recognition Letters, vol. 12, pp. 731-737, 1991 https://doi.org/10.1016/0167-8655(91)90070-3
  21. M. Sadiku and R. Kiem, 'Newton-Cotes rules for triple integral,' IEEE Proceedings. Sou-theastcon '90, vol. 2, pp. 471-475, 1990 https://doi.org/10.1109/SECON.1990.117858
  22. W. H. Press and et al., Numerical Recipies in C, 2nd ed. pp. 161-164, Cambridge Press, NY, 1992
  23. L. L. Biel and et al., 'A Crops and Soils Data Base For Scene Radiation Research,' Proc. Machine Process. of Remotely Sensed Data Symp., West Lafayette, Indiana, 1982