The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Study on Support Vector Machines Using Mathematical Programming

수리계획법을 이용한 서포트 벡터 기계 방법에 관한 연구

  • 윤민 (연세대학교 통계연구소) ;
  • 이학배 (연세대학교 응용통계학과)
  • Published : 2005.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

Machine learning has been extensively studied in recent years as effective tools in pattern classification problem. Although there have been several approaches to machine learning, we focus on the mathematical programming (in particular, multi-objective and goal programming; MOP/GP) approaches in this paper. Among them, Support Vector Machine (SVM) is gaining much popularity recently. In pattern classification problem with two class sets, the idea is to find a maximal margin separating hyperplane which gives the greatest separation between the classes in a high dimensional feature space. However, the idea of maximal margin separation is not quite new: in 1960's the multi-surface method (MSM) was suggested by Mangasarian. In 1980's, linear classifiers using goal programming were developed extensively. This paper proposes a new family of SVM using MOP/GP techniques, and discusses its effectiveness throughout several numerical experiments.

기계학습은 패턴분류의 한 도구로써 광범위하게 연구되고 있다. 기계학습 방법들 중에서 서포트 벡터 기계(Support Vector Machines)는 많은 분야에서 연구되어지는 것으로 이진 패턴 분류문제에서 고차원의 특징공간에서 두 집합들 사이에 가장 큰 분리를 제공하는 최대 여유도(margin)를 가지는 분리 초평면을 찾는 것이다. 최대 여유도의 분리의 개념에 기초하여 Mangasarian(1968)은 다중-표면 방법(multi-surface method)을 제안하였고, 1980년대에 목적 계획법을 이용한 방법들이 광범위하게 개발되었다. 본 논문에서는 다목적 계획법과 목적 계획법을 이용한 수리계획법인 서포트 벡터 기계의 두가지 방법들을 제안하고 수치 예제들을 통하여 효용성에 대하여 논의하고자 한다.

Keywords

References

  1. Asada, T. and Nakayama, H. (2003). Support vector machines using multi objective programming, in T. Tanino, T. Tanaka and M. Inuiguchi (eds.). Multi-objective Programming and Goal Programming, 93-98
  2. Bennett, K.P. and Mangasarian, O.L. (1992). Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software, 1, 23-34 https://doi.org/10.1080/10556789208805504
  3. Cavalier, T.M., Ignizio, J.P. and Soyster, A.L. (1989). Discriminant analysis via mathematical programming: certain problems and their causes. Computers and Operations Research, 16, 353-362 https://doi.org/10.1016/0305-0548(89)90007-5
  4. Cherkassky, V. and Mulier, F. (1998). Learning from Data Concepts, Theory, and Methods, John Wiley & Sons
  5. Cortes, C. and Vapnik, V. (1995). Support vector networks, Machine Learning, 20, 273-297
  6. Cristianini, N. and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge University Press
  7. Erenguc, S.S. and Koehler, G.J. (1990). Survey of mathematical programming models and experimental results for linear discriminant analysis, Managerial and Decision Economics, 11, 215-225 https://doi.org/10.1002/mde.4090110403
  8. Freed, N. and Glover, F. (1981). Simple but powerful goal programming models for discriminant problems, European Journal of Operational Research, 7, 44-60 https://doi.org/10.1016/0377-2217(81)90048-5
  9. Glover, F. (1990). Improved linear programming models for discriminant analysis, Decision Sciences, 21, 771-785 https://doi.org/10.1111/j.1540-5915.1990.tb01249.x
  10. Kuhn, H.W. and Tucker, A.W. (1951). Nonlinear programming, In Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probabilistic, University of California Press, 481-492
  11. Mangasarian, O.L. (1968). Multi surface method of pattern separation, IEEE Transaction on Information Theory, IT-14, 801-807
  12. Mangasarian, O.L. (1999). Arbitrary-norm separating plane, Operations Research Letters, 24, 15-23 https://doi.org/10.1016/S0167-6377(98)00049-2
  13. Marcotte, P. and Savard, G. (1992). Novel approaches to the discrimination problem, ZOR-Methods and Models of Operations Research, 36, 517-545 https://doi.org/10.1007/BF01416243
  14. Nakayama, H. and Asada, T. (2001). Support vector machines formulated as multi objective linear programming, Proceedings of ICOTA 2001, 3, 1171-1178
  15. Novikoff, A.B. (1962). On the convergence proofs on perceptrons, In Symposium on the Mathematical Theory of Automata, 12, 615-622. Polytechnic Institute of Brooklyn
  16. Sawaragi, Y, Nakayama, H. and Tanino, T. (1985). Theory of Multiobjective Optimization, Academic Press
  17. Scholkopf, B. and Smola, A.J. (1998). New support vector algorithms, NeuroCOLT2 Technical Report Series, NC2-TR-1998-031
  18. Yoon, M., Yun, YB. and Nakayama, H. (2004). Total margin algorithms in support vector machines, IEICE Transactions on Information and Systems, 87-D, 1223-1230
  19. http://www.ics.uci.edu/mlearn/MLSummary.html