Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

FPGA Implementation of SVM Engine for Training and Classification

기계학습 및 분류를 위한 SVM 엔진의 FPGA 구현

  • Na, Wonseob (Dept. of Electronics and Communications Engineering, Kwangwoon University) ;
  • Jeong, Yongjin (Dept. of Electronics and Communications Engineering, Kwangwoon University)
  • Received : 2016.11.30
  • Accepted : 2016.12.28
  • Published : 2016.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

SVM, a machine learning method, is widely used in image processing for it's excellent generalization performance. However, to add other data to the pre-trained data of the system, we need to train the entire system again. This procedure takes a lot of time, especially in embedded environment, and results in low performance of SVM. In this paper, we implemented an SVM trainer and classifier in an FPGA to solve this problem. We parlallelized the repeated operations inside SVM and modified the exponential operations of the kernel function to perform fixed point modelling. We implemented the proposed hardware on Xilinx ZC 706 evaluation board and used TSR algorithm to verify the FPGA result. It takes about 5 seconds for the proposed hardware to train 2,000 data samples and 16.54ms for classification for $1360{\times}800$ resolution in 100MHz frequency, respectively.

기계학습 방법의 하나인 SVM은 뛰어난 일반화 성능으로 영상처리 분야에서 많이 사용하고 있다. 하지만 SVM을 이용한 시스템에서 미리 학습된 데이터가 아닌 다른 데이터를 이용하려하면 새로 학습을 시켜야 하는 경우가 생긴다. 특히, 임베디드 환경에서는 이러한 상황에서 학습 시간이 오래 걸려 SVM을 적절히 이용하지 못하는 경우가 있다. 본 논문에서는 이러한 문제점을 해결하기 위하여 SVM의 학습 및 분류를 모두 수행할 수 있도록 하나의 FPGA로 구현하였다. SVM 연산의 복잡성으로 인해 생기는 반복연산을 병렬처리를 통하여 해결하고 커널 사용으로 생기는 지수 연산을 변형하여 고정 소수점 연산이 가능하도록 하였다. 제안하는 하드웨어는 Xilinx사의 ZC 706보드에 구현하였고, 구현한 FPGA의 검증을 위하여 TSR 알고리즘을 이용하였다. 구현한 하드웨어는 100 MHz의 주파수로 동작하며, 2천개의 데이터를 이용한 학습 시 약 5sec가 소요되고 $1360{\times}800$ 해상도에서 분류 시 약 16.54msec가 소요됨을 확인했다.

Keywords

References

  1. K. Grauman and T. Darrell, "The pyramid match kernel Discriminative classification with sets of image features", Proceedings of IEEE International Conference on Computer Vision, 2005
  2. Y. H. Kang, Y. B. Park, "Design of Automatic Document Classifier for IT documents based on SVM, " j.inst.Korean.electr.electron.eng, Vol. 8, No. 2 186-194, 2004
  3. J. Platt, "Fast Training of Support Vector Machines Using Sequential Minimal Optimization," in Advances in kernel methods - Support Vector Learning, MIT Press, 185-208, 1999
  4. C. Chang and C. Lin, "LIBSVM : A Library for Support Vector Machine," ACM Transactions on Intelligent Systems and Technology, Vol 2, No.3, 1-27, 2011
  5. S. Cadambi, I. Srihari, et al. "A massively parallel FPGA-based coprocessor for support vector machines," Field Programmable Custom Computing Machines '09. 17th IEEE Symposium on, Napa, USA, 2009
  6. T. Kuan, J. Wang, J. Wang, P. Lin, G. Gu, "VLSI Design of an SVM Learning Core on Sequential Minimal Optimization Algorithm," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 20, No. 4, 673-683, 2012 https://doi.org/10.1109/TVLSI.2011.2107533
  7. M. Qasaimeh, A. Sagahyroon, and T. Shanableh, "FPGA-Based Parallel Hardware Architecture for Real-Time Image Classification," Computational Imaging, IEEE Transactions, Vol. 1, No. 1 56-70, 2015 https://doi.org/10.1109/TCI.2015.2424077
  8. Marta Ruiz-Llata, Guillermo Guarnizo and Mar Yebenes-Calvino, "FPGA Implementation of a Support Vector Machine for Classification and Regression," IJCNN, 2010
  9. Xipeng Pan, Huihua Yang, Lingqiao Li, Zhenbing Liu, Le Hou , "FPGA Implementation of SVM Decision Function Based on Hardware-friendly Kernel," ICCIS, 2013
  10. Bryan Catanzaro, "Fast Support Vector Machine Training and Classification on Graphics Processors," IMAGING, Vol. 1, No. 1, March 2015
  11. Christos Kyrkou, Theocharis Theocharides, "A Parallel Hardware Architecture for Real-Time Object Detection with Support Vector Machines," IEEE TRANSACTIONS ON COMPUTERS, Vol. 61, No. 6, 2012
  12. Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin, "A Practical Guide to Support Vector Classification," Technical Report, Department of Computer Science National Taiwan University, 2003
  13. W. S. Na, S. W. Han, Y. J. Jeong, "FPGA Design of SVM Classifier for Real Time Image Processing," j.inst.Korean.electr.electron.eng, Vol. 20, No. 3 209-219, 2016
  14. C. Cortes and V. Vapnik, "Support-Vector Network," Machine Learning, Vol. 20, No. 3 273-297, 1995 https://doi.org/10.1007/BF00994018
  15. Il-Seok Oh, Pattern Recognition, Kyobomoongo, 137-173, 2008
  16. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning - Data Mining, Inference, and Prediction, 2008
  17. V. Vapnik, Estimation of Dependances Based on Empirical Data, Springer-Verlag, 1982
  18. E. Osuna, R. Freund, F. Girosi, "An improved training algorithm for support vector machines," Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Workshop, 276-285, 1997
  19. J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition," Neural Networks, vol. 32, pp. 323-332, 2012. https://doi.org/10.1016/j.neunet.2012.02.016
  20. Tomasz Kryjak, Mateusz Komorkiewicz, Marek Gorgon, "FPGA IMPLEMENTATION OF REAL-TIME HEAD-SHOULDER DETECTION USING LOCAL BINARY PATTERNS, SVM AND FOREGROUND OBJECT DETECTION," Design and Architectures for Signal and Image Processing (DASIP) Conference, Karlsruhe, Germany, 2012