ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Modified Fixed-Threshold SMO for 1-Slack Structural SVMs

  • Received : 2009.07.29
  • Accepted : 2009.10.28
  • Published : 2010.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we describe a modified fixed-threshold sequential minimal optimization (FSMO) for 1-slack structural support vector machine (SVM) problems. Because the modified FSMO uses the fact that the formulation of 1-slack structural SVMs has no bias, it breaks down the quadratic programming (QP) problems of 1-slack structural SVMs into a series of smallest QP problems, each involving only one variable. For various test sets, the modified FSMO is as accurate as existing structural SVM implementations (n-slack and 1-slack SVM-struct) but is faster on large data sets.

Keywords

References

  1. B. Taskar, C. Guestrin, and D. Koller, "Max Margin Markov Networks," NIPS, vol. 16, 2004.
  2. I. Tsochantaridis et al., "Support Vector Machine Learning for Interdependent and Structured Output Spaces," Proc. ICML, 2004, pp. 104.
  3. B. Taskar et al., "Max-Margin Parsing," Proc. EMNLP, 2004.
  4. C.N. Yu et al., "Support Vector Training of Protein Alignment Models," Proc. RECOMB, 2007, pp. 253-267.
  5. Y. Yue et al., "A Support Vector Method for Optimization Average Precision," Proc. SIGIR, 2007, pp. 271-278.
  6. J. Platt, "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines," Microsoft Research Technical Report MSR-TR-98-14, 1998.
  7. H. Kim and W. Kim, "Eye Detection in Facial Images Using Zernike Moments with SVM," ETRI J., vol. 30, no. 2, 2008, pp. 335-337. https://doi.org/10.4218/etrij.08.0207.0150
  8. M. Collins et al., "Exponentiated Gradient Algorithms for Conditional Random Fields and Max-Margin Markov Networks," JMLR, vol. 9, 2008, pp. 1775-1822.
  9. N.D. Ratliff, J.A. Bagnell, and M.A. Zinkevich, "(Online) Subgradient Methods for Structured Prediction," Proc. AISTATS, 2007.
  10. S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal Estimated Subgradient Solver for SVM," Proc. ICML, 2007, pp. 807-814.
  11. C. Lee and M. Jang, "Fast Training of Structured SVM Using Fixed-Threshold Sequential Minimal Optimization," ETRI J., vol. 31, no. 2, 2009, pp. 121-128. https://doi.org/10.4218/etrij.09.0108.0276
  12. C.H. Teo et al., "A Scalable Modular Convex Solver for Regularized Risk Minimization," Proc. KDD, 2007, pp. 727-736.
  13. T. Joachims, T. Finley, and C.N. Yu, "Cutting-Plane Training of Structural SVMs," MLJ, vol. 77, no. 1, 2009, pp. 27-59. https://doi.org/10.1258/rsmmlj.77.1.27
  14. T. Joachims, "Training Linear SVMs in Linear Time," Proc. KDD, 2006, pp. 217-226.
  15. R. Fan, P. Chen, and C. Lin, "Working Set Selection Using Second Order Information for Training Support Vector Machines," JMLR, vol. 6, 2005, pp. 1889-1918.
  16. T. Joachims, "A Statistical Learning Model of Text Classification with Support Vector Machines," Proc. SIGIR, 2001, pp. 128-136.
  17. C. Lin, "Asymptotic Convergence of an SMO Algorithm Without Any Assumptions," IEEE Trans. Neural Networks, vol. 13, no. 1, 2002, pp. 248-250. https://doi.org/10.1109/72.977319
  18. S. Keerthi and E. Gilbert, "Convergence of a Generalized SMO Algorithm for SVM Classifier Design," Machine Learning, vol. 46, no. 1-3, 2002, pp. 351-360. https://doi.org/10.1023/A:1012431217818
  19. V. Franc and S. Sonnenburg, "Optimized Cutting Plane Algorithm for Support Vector Machines," Proc. ICML, vol. 307, 2008, pp. 320-327.
  20. S. Keerthi et al. "A Sequential Dual Method for Large Scale Multi-Class Linear SVMs," Proc. KDD, 2008, pp. 408-416.
  21. Y. LeCun et al., "Gradient-Based Learning Applied to Document Recognition," Proceedings of the IEEE, vol. 86, no. 11, Nov. 1988, pp. 2278-2324.
  22. K. Lang. "Newsweeder: Learning to Filter Netnews," Proc. ICML, 1995.
  23. E.F.T.K. Sang and S. Buchholz. "Introduction to the CoNLL-2000 Shared Task: Chunking," Proc. CoNLL-2000 and LLL-2000, 2000, pp. 127-132.
  24. D. Lee, H. Rim, and D. Yook, "Automatic Word Spacing Using Probabilistic Models Based on Character n-Grams," IEEE Intelligent Systems, vol. 22, no. 1, 2007, pp. 28-35. https://doi.org/10.1109/MIS.2007.4

Cited by

  1. Large-Margin Training of Dependency Parsers Using Pegasos Algorithm vol.32, pp.3, 2010, https://doi.org/10.4218/etrij.10.0209.0483
  2. A Prior Model of Structural SVMs for Domain Adaptation vol.33, pp.5, 2010, https://doi.org/10.4218/etrij.11.0110.0571
  3. Analysis of the Empirical Effects of Contextual Matching Advertising for Online News vol.34, pp.2, 2010, https://doi.org/10.4218/etrij.12.0211.0171
  4. Finding more trustworthy answers: Various trustworthiness factors in question answering vol.39, pp.4, 2010, https://doi.org/10.1177/0165551513478893
  5. Domain-Adaptation Technique for Semantic Role Labeling with Structural Learning vol.36, pp.3, 2014, https://doi.org/10.4218/etrij.14.0113.0645
  6. Predicting the Lifespan and Retweet Times of Tweets Based on Multiple Feature Analysis vol.36, pp.3, 2010, https://doi.org/10.4218/etrij.14.0113.0657
  7. Structural SVM을 이용한 백과사전 문서 내 생략 문장성분 복원 vol.21, pp.2, 2015, https://doi.org/10.13088/jiis.2015.21.2.131
  8. Using Machine Learning Algorithms for Breast Cancer Risk Prediction and Diagnosis vol.83, pp.None, 2010, https://doi.org/10.1016/j.procs.2016.04.224
  9. A semantic annotation framework for scientific publications vol.51, pp.3, 2010, https://doi.org/10.1007/s11135-016-0369-3
  10. Fine-Grained Mobile Application Clustering Model Using Retrofitted Document Embedding vol.39, pp.4, 2010, https://doi.org/10.4218/etrij.17.0116.0936
  11. Reality mining and predictive analytics for building smart applications vol.6, pp.None, 2010, https://doi.org/10.1186/s40537-019-0227-y
  12. Hybrid Translation with Classification: Revisiting Rule-Based and Neural Machine Translation vol.9, pp.2, 2020, https://doi.org/10.3390/electronics9020201