Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Dependency-based Framework of Combining Multiple Experts for Recognizing Unconstrained Handwritten Numerals

무제약 필기 숫자를 인식하기 위한 다수 인식기를 결합하는 의존관계 기반의 프레임워크

  • Published : 2000.08.15
Download PDF
⟨ Previous Next ⟩

Abstract

Although Behavior-Knowledge Space (BKS) method, one of well known decision combination methods, does not need any assumptions in combining the multiple experts, it should theoretically build exponential storage spaces for storing and managing jointly observed K decisions from K experts. That is, combining K experts needs a (K+1)st-order probability distribution. However, it is well known that the distribution becomes unmanageable in storing and estimating, even for a small K. In order to overcome such weakness, it has been studied to decompose a probability distribution into a number of component distributions and to approximate the distribution with a product of the component distributions. One of such previous works is to apply a conditional independence assumption to the distribution. Another work is to approximate the distribution with a product of only first-order tree dependencies or second-order distributions as shown in [1]. In this paper, higher order dependency than the first-order is considered in approximating the distribution and a dependency-based framework is proposed to optimally approximate the (K+1)st-order probability distribution with a product set of dth-order dependencies where ($1{\le}d{\le}K$), and to combine multiple experts based on the product set using the Bayesian formalism. This framework was experimented and evaluated with a standardized CENPARMI data base.

K개의 인식기로부터 관찰된 K개 결정을 결합하는 결합 방법론 중의 하나인 BKS (Behavior-Knowledge Space) 방법은 아무런 가정 없이 이들 결정을 결합하지만, 관찰된 K개 결정을 저장하고 관리하려면 이론적으로 기하학적인 저장 공간을 만들어야 한다. 즉, K개의 인식기 결정을 결합하기 위하여 (K+1)차 확률 분포를 필요로 하는데, 작은 K라 할지라도 그 확률 분포를 저장하거나 평가하는 것이 어렵다는 것은 이미 잘 알려져 있다. 그러한 문제점을 극복하기 위해서는 고차 확률 분포를 몇 개의 구성 분포로 나누고, 이들 구성 분포의 곱(product)으로 고차 확률 분포를 근사시켜야 한다. 그러한 이전 방법 중의 하나는 그 확률 분포에 조건부 독립 가정을 적용하는 것이고, 다른 방법으로는 [1]에서와 같이 그 확률 분포를 단지 트리 의존관계 또는 2차 구성 분포의 곱으로 근사하는 것이다. 본 논문에서는, 구성 분포의 곱으로 근사하는 방법에서, 2차 이상의 고차 구성 분포까지 고려하여 (K+1)차 확률 분포를 d차 ($1{\le}d{\le}K$) 의존관계에 의한 최적의 곱으로 근사하고, 베이지안 방법과 그 곱을 기반으로 다수 인식기의 결정을 결합하는 의존관계 기반의 프레임워크를 제안한다. 이 프레임워크는 표준 CENPARMI 데이타베이스로 실험되어 평가되었다.

Keywords

References

  1. Chow, C. K. and Liu, C. N., 'Approximating Discrete Probability Distributions with Dependence Trees,' IEEE Trans. on Information Theory, 14(3):462-467, 1968 https://doi.org/10.1109/TIT.1968.1054142
  2. Hull, J. J., Commike, A., and Ho, T. K., 'Multiple algorithm for handwritten character recognition,' In Proceedings of the 1st Int. Workshop on Frontiers in Handwriting Recognition, pp. 117-129, 1990
  3. Suen, C. Y., Nadal, C., Mai, T. A., Legault, R., and Lam, L., 'Recognition of totally unconstrained handwritten numerals based on the concept of multiple experts,' In Proceedings of the 1st Int. Workshop on Frontiers in Handwriting Recognition, pp. 131-143, 1990
  4. Xu, L., Krzyzak, A., and Suen, C. Y., 'Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition,' IEEE Trans. on Systems, Man, and Cybernetics, 22(3):418-435, 1992 https://doi.org/10.1109/21.155943
  5. Huang, Y. S. and Suen, C. Y., 'A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals,' IEEE Trans. on Pattern Analysis and Machine Intelligence, 17(1):90-94, 1995 https://doi.org/10.1109/34.368145
  6. Mandler, E. and Schuermann, J., 'Combining the classification results of independent classifiers based on the dempster/shafer theory of evidence,' In E. S. Gelsema and L. N. Kanal, editors, Pattern Recognition and Artificial Intelligence, pp. 381-393. 1988
  7. Franke, J. and Mandler, E., 'A comparison of two approaches for combining the votes of cooperating classifiers,' In Proceedings of the 11th IAPR Int. Conference on Pattern Recognition, Vol.2, pp. 611-614, 1992 https://doi.org/10.1109/ICPR.1992.201786
  8. Lee, D.-S. and Srihari, S. N., 'Handprinted Digit Recognition : A Comparison of Algorithms,' In Proceedings of the 3rd Int. Workshop on Frontiers in Handwriting Recognition, pp. 153-162, 1993
  9. Kang, H.-J., Kim, K., and Kim, Jin H., 'Optimal Approximation of Discrete Probability Distribution with kth-order Dependency and Its Applications to Combining Multiple Classifiers,' Pattern Recognition Letters, 18(6), pp. 515-523, 1997 https://doi.org/10.1016/S0167-8655(97)00041-X
  10. Lewis, P. M., 'Approximating Probability Distributions to Reduce Storage Requirement,' Information and Control, 2:214-225, 1959. https://doi.org/10.1016/S0019-9958(59)90207-4
  11. Kang, H.-J. and Lee, S.-W., 'A Dependency-based Framework of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals,' In Proceedings of 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol.2, pp. 124-129, 1999 https://doi.org/10.1109/CVPR.1999.784619
  12. Ho, T. K., Hull J. J., and Srihari, S. N., 'Decision Combination of Multiple Classifier Systems,' IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(1), pp. 66-75, 1994 https://doi.org/10.1109/34.273716
  13. Woods, K., Kegelmeyer, W. P. Jr., and Bowyer, K., 'Combination of Multiple Classifiers Using Local Accuracy Estimates,' IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(4), pp. 405-410, 1997 https://doi.org/10.1109/34.588027
  14. Kittler, J., Hatef, M., Duin, R. P. W., and Matas, J., 'On Combining Classifiers,' IEEE Trans. on Pattern Analysis and Machine Intelligence, 20(3), pp. 226-239, 1998 https://doi.org/10.1109/34.667881