Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Prediction Performance of Hybrid Least Square Support Vector Machine with First Principle Knowledge

First Principle을 결합한 최소제곱 Support Vector Machine의 예측 능력

  • 김병주 (영산대학교 컴퓨터 정보공학부) ;
  • 심주용 (대구가톨릭대학교 정보통계학과) ;
  • 황창하 (대구가톨릭대학교 정보통계학과) ;
  • 김일곤 (경북대학교 컴퓨터과학과)
  • Published : 2003.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

A hybrid least square Support Vector Machine combined with First Principle(FP) knowledge is proposed. We compare hybrid least square Support Vector Machine(HLS-SVM) with early proposed models such as Hybrid Neural Network(HNN) and HNN with Extended Kalman Filter(HNN-EKF). In the training and validation stage HLS-SVM shows similar performance with HNN-EKF but better than HNN, whereas, in the testing stage, it shows three times better than HNN-EKF, hundred times better than HNN model.

본 논문에서는 최근 뛰어난 예측력으로 각광받는 최소제곱 Support Vector Machine(Least Square Support Vector Machine: LS-SVM)과 First Principle(FP)을 결합한 하이브리드 최소제곱ㆍSupport Vector Machine 모델, HLS-SVM(Hybrid Least Square-Super Vector Machine)을 제안한다. 제안한 모델인 하이브리드 최소제곱 Support Vector Machine을 기존의 방법인 하이브리드 신경망(Hybrid Neural Network:HNN), 비선형 칼만필터와 하이브리드 신경망을 결합한 HNN-EKF (Hybrid Neural Network with Extended Kalman Filter) 모델과 비교해 보았다. HLS-SVM 모델은 학습 및 validation 과정에서는 HNN-EKF와 근사한 성능을 보였고, HNN 보다는 우수한 결과를 보였고, 일반화 성능에서는 HNN-EKF에 비해 3배, HNN보다 100배정도 우수한 결과를 보였다.

Keywords

References

  1. S. Gunn, 'Support Vector Machines for Classification and Regression,' ISIS Technical Report, U. of Southampton, 1998
  2. E. Osuna, R. Freund, and F. Girosi, 'Support Vector Machines: Training and Applications,' Technical Report, MIT AI Laboratory, 1997
  3. http://www.support-vector.ws/html/downloads.html
  4. C. Brosilow, and M. Tong, 'Inference Control of Processes, Part II The Structure and Dynamics of Inferential Control Systems,' AIChE J.,Vol. 24, No. 3, 1978
  5. M. HADJISKI, Control of Indirect Measurable and Time-Varying Technological Plants, Ph. D. Thesis, HIChT, Sofia, Bulgaria, 1979
  6. B. JOSEPH, and C. BROSILOU, 'Inferential Control of Processes, Part I Steady State Analysis and Design,' AIChE J., Vol. 24, No. 3, 1978 https://doi.org/10.1002/aic.690240313
  7. http://www.ici.ro/ici/revista/sic99_1/art04.html
  8. De Veaux, R. Bain and L.H. Ungar, 'Hybrid Neural Network models for environmential process control,' Journal of ENVIRONMETRICS10, 225-236, 1999
  9. P. LINDSKOG, and L. LJUNG, 'Ensuring Certain Physical Properties in Black Box Model by Applying Fuzzy Techniques,' Technical Report, 1996
  10. R. Caruana, S. Lawrence and L. Giles, 'Overfitting in Neural Nets: Backpropagaton, Conjugate Gradient, and Early Stopping,' Neural Information Processing Systems, Denver, Colorado, November 28-30, 2000
  11. J.A.K. Suykens, 'Nonlinear Modeling and Support Vector Machines,' Accessible at http://www.kdiss.or.kr/kdiss/
  12. G. Weich, and G. Bishop, 'An Introduction to the Kalman Filter,' Siggraph Course8, 2001
  13. X. Shao, Model Selection Using Statistical Learning Theory, Ph. D. Thesis, U. of Minnesota, 1999
  14. V. Vapnik, The Nature of Statistical Learning Theory, Springer, 1995
  15. V. Vapnik, Statistical Learning Theory, Springer, 1998
  16. S. Gavin, C, Cawley and L.C. Talbot, 'Fast Exact Leave One Out Cross Validation of Least Squares Support Vector Machines,' European Symposium on Artificial Neural Networks, Bruges, Belgium, April 24-26, 2002