Pattern Recognition System Combining KNN rules and New Feature Weighting algorithm

KNN 규칙과 새로운 특징 가중치 알고리즘을 결합한 패턴 인식 시스템

  • Lee Hee-Sung (School of Electrical and Electronic Eng., Yonsei Univ.) ;
  • Kim Euntai (School of Electrical and Electronic Eng., Yonsei Univ.) ;
  • Kim Dongyeon (School of Electrical and Electronic Eng., Yonsei Univ.)
  • 이희성 (연세대학교 전기전자공학부) ;
  • 김은태 (연세대학교 전기전자공학부) ;
  • 김동연 (국립 한경대학교 전자공학과)
  • Published : 2005.07.01

Abstract

This paper proposes a new pattern recognition system combining the new adaptive feature weighting based on the genetic algorithm and the modified KNN(K Nearest-Neighbor) rules. The new feature weighting proposed herein avoids the overfitting and finds the Proper feature weighting value by determining the middle value of weights using GA. New GA operators are introduced to obtain the high performance of the system. Moreover, a class dependent feature weighting strategy is employed. Whilst the classical methods use the same feature space for all classes, the Proposed method uses a different feature space for each class. The KNN rule is modified to estimate the class of test pattern using adaptive feature space. Experiments were performed with the unconstrained handwritten numeral database of Concordia University in Canada to show the performance of the proposed method.

본 논문에서는 유전자 알고리즘을 이용한 새로운 적응적 특징 가중치 방식과 클래스별로 적용된 KNN(Nearest -Neighbor) 규칙을 이용한 새로운 패턴 인식 시스템을 제안한다. 패턴 인식 시스템의 성능을 향상시키기 위하여, 새로운 연산자를 갖는 유전자 알고리즘으로 가중치의 중간값을 결정함으로써 과잉 맞춤(overfitting)을 피하면서, 데이터의 분포에 따라 적절한 특징의 가중치를 찾는 새로운 특징 가중치 알고리즘을 제안한다. 또한, 제안하는 방법은 각각의 클래스를 가장 잘 표현하는 특징 공간들을 개별적으로 찾는다. KNN분류기는 클래스별로 찾은 특징 공간들을 이용하여 클래스에 따라 특징 공간을 변화시켜 미지 패턴의 클래스를 예측한다. 제안된 알고리즘은 Concordia대학의 handwritten numeral database에 적용시켜 그 성능을 확인하였다.

Keywords

References

  1. A. Jain and D. Zongker, 'Feature selection: Evaluation, application, and small sample performance,' EEE Trans. on Pattern Anal. and Machine Intell. vol. 19, pp. 153-158, Feb. 1997 https://doi.org/10.1109/34.574797
  2. L. Davis, Handbook of Genetic Algorithms. Van Nostrand Reinhold, 1991
  3. E. Gose, R. Johnsonbaugh, and S. Jost, Patten Recognition and Image Analysis. Prentice Hall PTR, 1996
  4. W. Siedlecki and J. Sklansky, 'A note on genetic algorithms for large-scale feature selection,' Pattern Recognition Letter, vol. 10, pp. 335-347, 1989 https://doi.org/10.1016/0167-8655(89)90037-8
  5. J. D. Kelly and L. Davis, 'Hybridizing the genetic algorithm and the k nearest neighbors classification algorithm,' in Proc. 4th Int. Conf. Genetic Algorithms, Appl., 1991, pp. 377-383
  6. Kohavi, R. Langlry, and Y. Yun, 'The utility of feature weighting in nearest neighbor algorithms,' European Conference on Machine Learning, ECML'97. 1997
  7. W. K. Pratt, Digital Image Processing. New York: Wiley, 1978
  8. S. Lee, 'Off-Line Recognition of Totally Unconstrained Handwritten Numerals Using Multilayer Cluster Neural Network,' IEEE Trans. on Pattern Anal. and Machine Intell., Vol. 18, pp. 648-652, June 1996 https://doi.org/10.1109/34.506416