Nonlinear Feature Extraction using Class-augmented Kernel PCA

클래스가 부가된 커널 주성분분석을 이용한 비선형 특징추출

  • Park, Myoung-Soo (KIST, Human-centered Interaction and Robotics Research Center) ;
  • Oh, Sang-Rok (KIST, Human-centered Interaction and Robotics Research Center)
  • 박명수 (한국과학기술연구원 실감교류로보틱스연구센터) ;
  • 오상록 (한국과학기술연구원 실감교류로보틱스연구센터)
  • Received : 2011.04.22
  • Accepted : 2011.08.26
  • Published : 2011.09.25


In this papwer, we propose a new feature extraction method, named as Class-augmented Kernel Principal Component Analysis (CA-KPCA), which can extract nonlinear features for classification. Among the subspace method that was being widely used for feature extraction, Class-augmented Principal Component Analysis (CA-PCA) is a recently one that can extract features for a accurate classification without computational difficulties of other methods such as Linear Discriminant Analysis (LDA). However, the features extracted by CA-PCA is still restricted to be in a linear subspace of the original data space, which limites the use of this method for various problems requiring nonlinear features. To resolve this limitation, we apply a kernel trick to develop a new version of CA-PCA to extract nonlinear features, and evaluate its performance by experiments using data sets in the UCI Machine Learning Repository.

본 논문에서는 자료패턴을 분류하기에 적합한 특징을 추출하는 방법인, 클래스가 부가된 커널 주성분분석(class-augmented kernel principal component analysis)를 새로이 제안하였다. 특징추출에 널리 이용되는 부분공간 기법 중, 최근 제안된 클래스가 부가된 주성분분석(class-augmented principal component analysis)은 패턴 분류를 위한 특징을 추출하기 위해 이용되는 선형분류분석(linear discriminant analysis)등에 비해 정확한 특징을 계산상의 문제 없이 추출할 수 있는 기법이다. 그러나, 추출되는 특징은 입력의 선형조합으로 제한되어 자료에 따라 적절한 특징을 추출하기 어려운 경우가 발생한다. 이를 해결하기 위하여 클래스가 부가된 주성분분석에 커널 트릭을 적용하여 비선형 특징을 추출할 수 있는 새로운 부분공간 기법으로 확장하고, 실험을 통하여 성능을 평가하였다.



  1. I. T. Jolliffe, Principal Component Analysis, Springer-Verlag, 1986.
  2. K. Fukunaga, Introduction to Statistical Pattern Recognition, Morgan-Kaufmann, 1990.
  3. M. S. Park and J. Y. Choi, "Feature Extraction Using Class-Augmented Principal Component Analysis (CA-PCA)", Lecture Notes in Computer Science, Vol. 4132, pp. 606-615, 2006.
  4. M. S. Park and J. Y. Choi, "Theoretical Analysis On Feature Extraction Capability Of Class-Augmented PCA", Pattern Recognition, Vol. 42, Issue 11, pp. 2353-2362, 2009.
  5. B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem", Neural Computation, Vol. 10, No. 5, pp. 1299-1319, 1998.
  6. S. Mika, G. Ratsch, J. Weston, B. Schölkopf, and K.-R. Müller, "Fisher Discriminant Analysis With Kernels", in Proceedings of 1999 IEEE Signal Processing Society Workshop on Nerual Networks for Signal Proceeding, Vol. 9, pp. 41-48, 1999.
  7. G. Baudat and F. Anouar, "Generalizaed Discriminant Analysis Using A Kernel Approach", Neural Computation, vol. 12, pp. 2385-2404, 2000.
  8. P. Cui and J. Fang, "KPCA Plus FDA For Fault Detection", Lecture Notes in Computer Science, Vol. 4493, pp. 597-606, 2007.
  9. UCI Machine Learning Repository website: