Journal of the Korean Operations Research and Management Science Society (한국경영과학회지)

The Korean Operations Research and Management Science Society (한국경영과학회)
권호 표지

A Branch-and-Bound Algorithm for Finding an Optimal Solution of Transductive Support Vector Machines

Transductive SVM을 위한 분지-한계 알고리즘

  • Published : 2006.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

Transductive Support Vector Machine(TSVM) is one of semi-supervised learning algorithms which exploit the domain structure of the whole data by considering labeled and unlabeled data together. Although it was proposed several years ago, there has been no efficient algorithm which can handle problems with more than hundreds of training examples. In this paper, we propose an efficient branch-and-bound algorithm which can solve large-scale TSVM problems with thousands of training examples. The proposed algorithm uses two bounding techniques: min-cut bound and reduced SVM bound. The min-cut bound is derived from a capacitated graph whose cuts represent a lower bound to the optimal objective function value of the dual problem. The reduced SVM bound is obtained by constructing the SVM problem with only labeled data. Experimental results show that the accuracy rate of TSVM can be significantly improved by learning from the optimal solution of TSVM, rather than an approximated solution.

Keywords

References

  1. Bennett K.P. and A. Demiriz, 'Semi-supervised support vector machines,' Advances in Neural Information Processing Systems, Vol.10(1998), MIT Press
  2. Bie, T. De and N. Christianini, 'Convex Methods for Transduction,' Advances in Neural Information Processing Systems, Vol. 16(2004), MIT Press
  3. Blum, A. and S. Chawla, 'Learning from Labeled and Unlabeled Data using Graph Mincuts,' Proceedings of the 18th International Conference on Machine Learning (ICML), 2001
  4. Chang, C.-C. and C.-J. Lin, LIBSVM：A Library for Support Vector Machines, http: //www.csie.ntu.edu.tw/-cjlin/libsvm, 2001
  5. Chapelle, O., J. Weston, and B. Sch${'{o}}$kopf, 'Cluster Kernels for Semi-supervised Learning,' Advances in Neural Information Processing Systems, Vol.15(2003), MIT Press
  6. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines and other Kernel-based Learning Methods, Cambridge University Press, 2000
  7. Joachims, T., 'Transductive Learning via Spectral Graph Partitioning,' Proceedings of the 20th International Conference on Machine Learning(ICML), 2003
  8. Joachims, T., 'Transductive Inference for Text Classification using Support Vector Machines,' Proceedings of the 20th International Conference on Machine Learning (ICML), 1999
  9. Nilsson, N.J., Introduction to Machine Learning, http://robotics.stanford.edu/people/ nilsson/mlbook.html
  10. Platt, J., 'Fast Training of Support Vector Machines using Sequential Minimal Optimization,' In B. Schokopf, C.J.C. Burges, and A.J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pp. 185-208
  11. Schokopf, B. and A.J. Smola, Learning with Kernels：Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, 2002
  12. Seeger, M., Learning with Labeled and Unlabeled Data, Technical Report, Institute for Adaptive and Neural Computation, University of Edinburgh, 2001
  13. Shi, J. and J. Malik, 'Normalized Cuts and Image Segmentation,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22(2000), pp.888-905 https://doi.org/10.1109/34.868688
  14. Szummer, M. and T. Jaakkola, 'Partially Labeled Classification with Markov Random Walks,' Advances in Neural Information Processing Systems, Vol.14(2002), MIT Press
  15. Vapnik, V., Statistical Learning Theory, Wiley, 1998
  16. Yu, S.X. and J. Shi, 'Grouping with Bias,' Advances in Neural Information Processing Systems, Vol.14(2001), MIT Press
  17. Zhu, X., Z. Ghahramani, and J. Lafferty, 'Semi-supervised Learning using Gaussian Fields and Harmonic Functions,' Proceedings of the 20th International Conference on Machine Learning(ICML), 2003