Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

A Real-Time Hardware Design of CNN for Vehicle Detection

차량 검출용 CNN 분류기의 실시간 처리를 위한 하드웨어 설계

  • Bang, Ji-Won (Dept. of Electronics & Communications Engineering, KwangWoon University) ;
  • Jeong, Yong-Jin (Dept. of Electronics & Communications Engineering, KwangWoon University)
  • Received : 2016.11.30
  • Accepted : 2016.12.28
  • Published : 2016.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, machine learning algorithms, especially deep learning-based algorithms, have been receiving attention due to its high classification performance. Among the algorithms, Convolutional Neural Network(CNN) is known to be efficient for image processing tasks used for Advanced Driver Assistance Systems(ADAS). However, it is difficult to achieve real-time processing for CNN in vehicle embedded software environment due to the repeated operations contained in each layer of CNN. In this paper, we propose a hardware accelerator which enhances the execution time of CNN by parallelizing the repeated operations such as convolution. Xilinx ZC706 evaluation board is used to verify the performance of the proposed accelerator. For $36{\times}36$ input images, the hardware execution time of CNN is 2.812ms in 100MHz clock frequency and shows that our hardware can be executed in real-time.

최근 딥 러닝을 중심으로 빠르게 발전하고 있는 기계학습 분류 알고리즘은 기존의 방법들보다 뛰어난 성능으로 인하여 주목받고 있다. 딥 러닝 중에서도 Convolutional Neural Network(CNN)는 영상처리에 뛰어나 첨단 운전자 보조 시스템(Advanced Driver Assistance System : ADAS)에서 많이 사용되고 있는 추세이다. 하지만 차량용 임베디드 환경에서 CNN을 소프트웨어로 동작시켰을 때는 각 Layer마다 연산이 반복되는 알고리즘의 특성으로 인해 수행시간이 길어져 실시간 처리가 어렵다. 본 논문에서는 임베디드 환경에서 CNN의 실시간 처리를 위하여 Convolution 연산 및 기타 연산들을 병렬로 처리하여 CNN의 속도를 향상시키는 하드웨어 구조를 제안한다. 제안하는 하드웨어의 성능을 검증하기 위하여 Xilinx ZC706 FPGA 보드를 이용하였다. 입력 영상은 $36{\times}36$ 크기이며, 동작주파수 100MHz에서 하드웨어 수행시간은 약 2.812ms로 실시간 처리가 가능함을 확인했다.

Keywords

References

  1. K. Grauman and T. Darrell, "The pyramid match kernel Discriminative classification with sets of image features," Proceedings of IEEE International Conference on Computer Vision, 2005.
  2. C. Cortes and V. Vapnik, "Support-Vector Network," Machine Learning, Vol. 20, No. 3 273-297, 1995. https://doi.org/10.1007/BF00994018
  3. Quinlan, J. Ross, C4. 5: programs for machine learning, Elsevier, 2014.
  4. Viola, Paul, and Michael J. Jones. "Robust real-time face detection," International journal of computer vision 57.2 pp. 137-154, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  5. Schmidhuber, Jurgen. "Deep learning in neural networks: An overview," Neural Networks 61 pp. 85-117, 2015. https://doi.org/10.1016/j.neunet.2014.09.003
  6. Dollar, Piotr, et al. "Pedestrian detection: An evaluation of the state of the art," IEEE transactions on pattern analysis and machine intelligence Vol. 34 No. 4 pp. 743-761, 2012. https://doi.org/10.1109/TPAMI.2011.155
  7. Y. B. Jo, et al. "Traffic Sign Recognition using SVM and Decision Tree for Poor Driving Environment," j.inst.Korean.electr.electron.eng Vol. 18 No.4 pp. 485-494, 2014.
  8. Salari, E., and D. Ouyang. "Camera-based Forward Collision and lane departure warning systems using SVM," 2013 IEEE 56th International Midwest Symposium on Circuits and Systems (MWSCAS), IEEE, 2013.
  9. J. Wu and X. Zhang, "A PCA classifier and its application in vehicle detection," in Proc. Int. Joint Conf. Neural Netw., vol. 1, pp. 600-604, 2001.
  10. Z. Sun, G. Bebis, and R. Miller, "On-road vehicle detection using Gabor filters and support vector machines," in Proc. 14th Int. Conf. Digit. Signal Process, pp. 1019-1022, 2002.
  11. Z. Sun, G. Bebis, and R. Miller, "Quantized wavelet features and support vector machines for on-road vehicle detection," in Proc. 7th Int. Conf. Control, Autom., Robot. Vis, pp. 1641-1646, 2002.
  12. Z. Sun, G. Bebis, and R. Miller, "Improving the performance of on-road vehicle detection by combining Gabor and wavelet features," in Proc. IEEE 5th Int. Conf. Intell. Transp. Syst, pp. 130-135, 2002.
  13. R. Lienhart and J. Maydt, "An extended set of Haar-like features for rapid object detection," in Proc. Int. Conf. Image Process, pp. 900-903, Jan. 2002.
  14. R. Lienhart, A. Kuranov, and V. Pisarevsky, "Empirical analysis of detection cascades of boosted classifiers for rapid object detection," in Proc. 25th German Pattern Recognit. Symp. pp. 297-304, 2003.
  15. Wen, Xuezhi, and Yuhui Zheng. "An improved algorithm based on AdaBoost for vehicle recognition," The 2nd International Conference on Information Science and Engine ering. IEEE, 2010.
  16. X. Wen et. al, "Efficient Feature Selection and Classification for Vehicle Detection," IEEE Trans. On Circuits and Systems for Video Technology, Vol.25, No.3, pp.508-517, Mar. 2015. https://doi.org/10.1109/TCSVT.2014.2358031
  17. O. Russakovsky et. al, "ImageNet Large Sclae Visual Recognition Challenge," International Journal of Computer Vision, Vol.115, No.3, pp.211-252, Dec.2015. https://doi.org/10.1007/s11263-015-0816-y
  18. LeCun, Yann, et al. "Backpropagation applied to handwritten zip code recognition," Neural computation, Vol 1. No. 4: pp. 541-551, 1989. https://doi.org/10.1162/neco.1989.1.4.541
  19. LeCun, Yann, et al. "Gradient-based learning applied to document recognition," Proceedings of the IEEE 86.11 pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  20. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems (NIPS), 2012.
  21. Gao, Yongbin, and Hyo Jong Lee. "Vehicle Make Recognition Based on Convolutional Neural Network," Information Science and Security (ICISS), 2015 2nd International Conference on. IEEE, 2015.
  22. LANGE, Stefan; ULBRICH, Fritz; GOEHRING, Daniel, "Online vehicle detection using deep neural networks and lidar based preselected image patches," Intelligent Vehicles Symposium (IV), pp. 954-959, 2016 IEEE.
  23. Bautista, Carlo Migel, et al. "Convolutional neural network for vehicle detection in low resolution traffic videos," Region 10 Symposium (TENSYMP), 2016 IEEE. IEEE, 2016.
  24. Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W.,Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu,T., Cheng-Yue, R., Mujica, F., Coates, A., and Ng, A. Y. An "Empirical Evaluation of Deep Learning on Highway Driving," ArXiv e-prints, April 2015.
  25. Jung, Seokwoo, et al. "Real-time Traffic Sign Recognition system with deep convolutional neural network," Ubiquitous Robots and Ambient Intelligence (URAI), 2016 13th International Conference on. IEEE, 2016.
  26. Fukui, Hiroshi, et al. "Pedestrian detection based on deep convolutional neural network with ensemble inference network," 2015 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2015.
  27. Li, Huimin, et al. "A high performance FPGA-based accelerator for large-scale convolutional neural networks," Field Programmable Logic and Applications (FPL), 2016 26th International Conference on. EPFL, 2016.
  28. Li, Ning, et al, "A multistage dataflow implementation of a Deep Convolutional Neural Network based on FPGA for high-speed object recognition," 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), IEEE, 2016.
  29. Linjie Yang, Ping Luo, Chen Change Loy, Xiaoou Tang. "A Large-Scale Car Dataset for Fine-Grained Categorization and Verification," Computer Vision and Pattern Recognition (CVPR), 2015.
  30. J. Arrospide, L. Salgado, M. Nieto, "Video analysis based vehicle detection and tracking using an MCMC sampling framework," EURASIP Journal on Advances in Signal Processing, vol. 2012, Article ID 2012:2, Jan. 2012.