Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Context Dependent Fusion with Support Vector Machines

Support Vector Machine을 이용한 문맥 민감형 융합

  • Received : 2013.02.21
  • Accepted : 2013.06.23
  • Published : 2013.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Context dependent fusion (CDF) is a fusion algorithm that combines multiple outputs from different classifiers to achieve better performance. CDF tries to divide the problem context into several homogeneous sub-contexts and to fuse data locally with respect to each sub-context. CDF showed better performance than existing methods, however, it is sensitive to noise due to the large number of parameters optimized and the innate linearity limits the application of CDF. In this paper, a variant of CDF using support vector machines (SVMs) for fusion and kernel principal component analysis (K-PCA) for context extraction is proposed to solve the problems in CDF, named CDF-SVM. Kernel PCA can shape irregular clusters including elliptical ones through the non-linear kernel transformation and SVM can draw a non-linear decision boundary. Regularization terms is also included in the objective function of CDF-SVM to mitigate the noise sensitivity in CDF. CDF-SVM showed better performance than CDF and its variants, which is demonstrated through the experiments with a landmine data set.

문맥 종속형 융합(CDF, Context Dependent Fusion)은 여러 분류기의 결과를 종합하여 성능을 향상시키는 융합 방법으로 주어진 문제의 문맥을 균일한 여러 문맥으로 나누고 각 문맥에서 문맥 종속적인 융합을 시도함으로써 기존 융합 방법에 비해 향상된 성능을 보여주었다. 하지만 CDF는 학습해야할 파라미터의 개수가 많아 학습 데이터가 적은 경우 잡음에 민감한 문제점이 있으며, 선형 알고리듬이라는 한계로 인해 문맥 추출 및 지역적 융합 과정에서 성능 저하의 원인이 된다. 본 논문에서는 CDF의 문제점을 완화할 수 있는 방법으로 SVM(Support Vector Machine)과 커널 주성분 분석을 이용한 CDF-SVM을 제안하였다. 커널 주성분 분석은 입력 벡터에 비선형 변환을 가함으로써 타원형이 아닌 비정형의 클러스터 생성이 가능하도록 해주며, SVM은 융합과정에서 비선형 경계의 생성을 가능하게 해주어 CDF의 선형성 제약을 극복하도록 해준다. 또한 목적함수에 정규화 항을 추가함으로써 잡음 민감성을 줄이도록 하였다. 제안한 CDF-SVM은 기존 CDF 및 그 변형들에 비해 나은 성능을 보여주었으며 이는 실험 결과를 통해 확인할 수 있다.

Keywords

References

  1. C.M. Bishop, "Pattern Recognition and Machine Learning," Springer, Singapore, 2007.
  2. M. Minsky, "Logical versus analogical or symbolic versus connectionist or neat versus scruffy," AI Magazine, Vol. 12, No. 2, pp. 34-51, Jun. 1991.
  3. L.I. Kuncheva, "Combining Pattern Classifiers," Wiley-Interscience, New York, 2004.
  4. L.I. Kuncheva, "Clustering-and-selection model for classifier combination," Proceedings of the 4th International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, pp. 185-188, 2000.
  5. R. Liu and B. Yuan, "Multiple classifiers combination by clustering and selection," Information Fusion, Vol. 2, No. 3, pp. 163-168, Sep. 2001. https://doi.org/10.1016/S1566-2535(01)00033-1
  6. H. Frigui, P. Gader, and A.C.B. Abdallah, "A generic framework for context-dependent fusion with application to landmine detection," Proceedings of SPIE Defense and Security Symposium, pp. 490-495, 2008.
  7. Gyeongyong Heo, Paul Gader, and Hichem Frigui, "A Noise Robust Variant of Context Extraction for Local Fusion," Proceedings of the 2010 IEEE International Conference on Fuzzy Systems, pp. 1-6, 2010.
  8. W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, "Numerical Recipes: The Art of Scientific Computing," Cambridge University Press, New York, 3rd Ed. 2007.
  9. N.R. Pal, K. Pal, J.M. Keller, and J.C. Bezdek, "A Possibilistic Fuzzy c-Means Clustering Algorithm," IEEE Transactions on Fuzzy Systems, Vol. 13, No. 4, pp. 517-530, Aug. 2005. https://doi.org/10.1109/TFUZZ.2004.840099
  10. M. Filippone, F. Camastra, F. Masulli, and S. Rovetta, "A survey of kernel and spectral methods for clustering," Pattern Recognition, Vol. 41, No. 1, pp. 176-190, Jan. 2008. https://doi.org/10.1016/j.patcog.2007.05.018
  11. Gyeongyong Heo, Paul Gader, and Hichem Frigui, "RKF-PCA: Robust Kernel Fuzzy PCA," Neural Networks, Vol. 22, No. 5-6, pp. 642-650, Jul.-Aug. 2009. https://doi.org/10.1016/j.neunet.2009.06.013
  12. V. Vapnik, "Statistical Learning Theory," John Wiley & Sons, New York, 1998.
  13. C.-F. Lin and S.-D. Wang, "Fuzzy support vector machines," IEEE Transactions on Neural Networks, vol. 13, no. 2, pp. 464-471, Mar. 2002. https://doi.org/10.1109/72.991432
  14. R. Mayer, F. Bucholtz, and D. Scribner, "Object detection by using whitening/dewhitening to transform target signatures in multitemporal hyperspectral and multispectral imagery," IEEE Transactions on Geoscience and Remote Sensing, Vol. 41, No. 5, pp. 1136-1142, May 2003. https://doi.org/10.1109/TGRS.2003.813553
  15. R.H. Yuhas, A.F. Goetz, and J.W. Boardman, "Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm," Summaries of the 3rd Annual JPL Airborne Geoscience Workshop, pp. 147-149, 1992.
  16. O.A. de Carvalho and P.R. Meneses, "Spectral correlation mapper (SCM): An improvement on the spectral angle mapper (SAM)," Summaries of the 9th Annual JPL Airborne Earth Science Workshop, pp. 65-74, 2000.

Cited by

  1. Study on Fault Detection of a Gas Pressure Regulator Based on Machine Learning Algorithms vol.25, pp.4, 2013, https://doi.org/10.9708/jksci.2020.25.04.019