Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Input Variables Selection by Principal Component Analysis and Mutual Information Estimation

주요성분분석과 상호정보 추정에 의한 입력변수선택

  • 조용현 (대구가톨릭대학교 컴퓨터정보통신공학부) ;
  • 홍성준 (대구가톨릭대학교 컴퓨터정보통신공학부)
  • Published : 2007.04.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents an efficient input variable selection method using both principal component analysis(PCA) and adaptive partition mutual information(AP-MI) estimation. PCA which is based on 2nd order statistics, is applied to prevent a overestimation by quickly removing the dependence between input variables. AP-MI estimation is also applied to estimate an accurate dependence information by equally partitioning the samples of input variable for calculating the probability density function. The proposed method has been applied to 2 problems for selecting the input variables, which are the 7 artificial signals of 500 samples and the 24 environmental pollution signals of 55 samples, respectively. The experimental results show that the proposed methods has a fast and accurate selection performance. The proposed method has also respectively better performance than AP-MI estimation without the PCA and regular partition MI estimation.

본 논문에서는 주요성분분석과 상호정보 추정을 조합한 입력변수선택 기법을 제안하였다. 여기서 주요성분분석은 2차원 통계성에 기반을 둔 기법으로 입력변수 간의 종속성을 빠르게 제거하여 과추정을 방지하기 위함이고, 상호정보의 추정은 적응적 분할을 이용하여 입력변수의 확률밀도함수를 계산함으로써 변수상호간의 종속성을 좀 더 정확하게 측정하기 위함이다. 제안된 기법을 각 500개 샘플의 7개 신호를 가지는 인위적인 문제와 각 55개 샘플의 24개의 신호를 가지는 환경오염신호를 대상으로 각각 실험한 결과, 빠르고 정확한 변수의 선택이 이루어짐을 확인하였다. 또한 주요성분분석을 수행하지 않을 때와 정규분할의 상호정보 추정 때보다 제안된 방법은 각각 우수한 선택성능이 있음을 확인하였다.

Keywords

References

  1. T. Trappenberg, J Ouyang, and A. Back, 'Input Variable Selection : Mutual Information and Linear Mixing Measures', IEEE Transactions on Knowledge and Data Engineering, Vol. 1, No.8, pp. 37-46, Jan. 2006
  2. A. Back and A. Cichocki, 'Input Variable Selection Using Independent Component Analysis and Higher Order Statistics', Proc. of ICA99, Jan. 1999
  3. A. Back and T. Trappenberg. 'Input Variable Selection Using Independent Component Analysis,' International joint Conference on Neural Networks, pp. 1-5, Washington, 1999
  4. A. Back and T. Trappenberg, 'Selecting Inputs for Modelling Using Normalized Higher Order Statistics and Independent Component Analysis,' IEEE Transactions on Neural Networks, Vol. 12, No.3, pp. 612-617, March. 2001 https://doi.org/10.1109/72.925564
  5. K. I. Diamantaras and S. Y. Kung, Principal Component Neural Networks : Theory and Applications, Adaptive and learning Systems for Signal Processing, Communications, and Control, John Wiley & Sons, Inc., 1996
  6. S. Haykin, Neural Networks : A Comprehensive Foundation, Prentice-Hall, 2ed, London, 1999
  7. A. Cichock and R. Unbehaun, Neural Networks for Optimization and Signal Processing. John Wiley & Sons., New York, 1993
  8. P. Foldiak, 'Adaptive Network for Optimal Linear Feature Extraction,' International Joint Conference on Neural Networks, Washington D.C., Vol. 1, pp. 401-406, June 1989