IEIE Transactions on Smart Processing and Computing

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Real-time Speed Limit Traffic Sign Detection System for Robust Automotive Environments

  • Hoang, Anh-Tuan (Research Institute for Nanodevice and Bio Systems, Hiroshima University) ;
  • Koide, Tetsushi (Research Institute for Nanodevice and Bio Systems, Hiroshima University) ;
  • Yamamoto, Masaharu (Research Institute for Nanodevice and Bio Systems, Hiroshima University)
  • Received : 2015.07.15
  • Accepted : 2015.08.24
  • Published : 2015.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper describes a hardware-oriented algorithm and its conceptual implementation in a real-time speed limit traffic sign detection system on an automotive-oriented field-programmable gate array (FPGA). It solves the training and color dependence problems found in other research, which saw reduced recognition accuracy under unlearned conditions when color has changed. The algorithm is applicable to various platforms, such as color or grayscale cameras, high-resolution (4K) or low-resolution (VGA) cameras, and high-end or low-end FPGAs. It is also robust under various conditions, such as daytime, night time, and on rainy nights, and is adaptable to various countries' speed limit traffic sign systems. The speed limit traffic sign candidates on each grayscale video frame are detected through two simple computational stages using global luminosity and local pixel direction. Pipeline implementation using results-sharing on overlap, application of a RAM-based shift register, and optimization of scan window sizes results in a small but high-performance implementation. The proposed system matches the processing speed requirement for a 60 fps system. The speed limit traffic sign recognition system achieves better than 98% accuracy in detection and recognition, even under difficult conditions such as rainy nights, and is implementable on the low-end, low-cost Xilinx Zynq automotive Z7020 FPGA.

Keywords

References

  1. Han, Y.: Real-time traffic sign recognition based on Zynq FPGA and ARM SoCs, Proc. 2014 IEEE Intl. Conf. Electro/Information Technology (EIT), pp. 373-376, 2014.
  2. Hoang, A.T., Yamamoto, M., Koide, T.: Low cost hardware implementation for traffic sign detection system, Proc. 2014 IEEE Asia Pacific Conf. Circuits and Systems (APCCAS2014), pp. 363-366, 2014.
  3. Irmak, M.: Real time traffic sign recognition system on FPGA, Master Thesis, The Graduate School of Natural and Applied Sciences of Middle East Technical University, 2010.
  4. Ishizuka, Y., and Hirai, Y.: Recognition system of road traffic signs using opponent-color filter, Technical report of IEICE, No. 103, pp. 13-18, 2004, (in Japanese).
  5. Keller, C. G., et al.: Real-time recognition of U.S. speed signs, Proc. Intl. IEEE Intelligent Vehicles Symposium, pp. 518-523, 2008.
  6. Liu, W., et al.: Real-time speed limit sign detection and recognition from image sequences, Proc. 2014 IEEE Intl. Conf. Artificial Intelligence and Computational Intelligence (AICI), pp. 262-267, 2010.
  7. Miura, J., et al.: An active vision system for realtime traffic sign recognition, Proc. Intl. IEEE Conf. Intelligent Transportation Systems, pp. 52-57, 2000.
  8. Moutarde, F., et al.: Robust on-vehicle real-time visual detection of American and European speed limit signs, with a modular traffic signs recognition system, Proc. Intl. IEEE Intelligent Vehicles Symposium, pp. 1122-1126, 2007.
  9. Muller, M., et al.: Design of an automotive traffic sign recognition system targeting a multi-core SoC implementation, Proc. Design, Automation & Test in Europe Conference & Exhibition 2010, pp. 532-537, 2010.
  10. Ozcelik, P. M., et al.: A template-based approach for real-time speed limit sign recognition on an embedded system using GPU computing, Proc. 32nd DAGM Conf. Pattern Recognition, pp. 162-171, 2010.
  11. Raiyn, J.,and Toledo, T.: Real-time road traffic anomaly detection, Journal of Transportation Technologies, 2014, No. 4, pp. 256-266, 2014.
  12. Schewior, G., et al.: A hardware accelerated configurable ASIP architecture for embedded realtime video-based driver assistance applications, Proc. 2011 IEEE Intl. Conf. Embedded Computer Systems (SAMOS), pp. 18-21, 2011.
  13. Soendoro, D., and Supriana, I.: Traffic sign recognition with color based method, shape-arc estimation and SVM, Proc. 2014 IEEE Intl. Conf. Electrical Engi-neering and Informatics (ICEEI), pp. 1-6, 2011.
  14. Souani, C., Faiedh, H., and Besbes, K.: Efficient algorithm for automatic road sign recognition and its hardware implementation, Journal of Real-Time Image Processing, Vol. 9, Issue 1, pp. 79-93, 2014. https://doi.org/10.1007/s11554-013-0348-z
  15. Takagi, M., and Fujiyoshi, H.: Road sign recognition using SIFT feature, Proc. 18th Symposium on Sensing via Image Information, LD2-06, 2007, (in Japanese).
  16. Torresen, J., et al.: Efficient recognition of speed limit signs, Proc. 7th Intl. IEEE Conf. Intelligent Transportation Systems, pp. 652-656, 2004.
  17. Waite, S. and Oruklu, E.: FPGA-based traffic sign recognition for Advanced Driver Assistance Systems, Journal of Transportation Technologies, Vol. 3, No. 1, pp. 1-16, 2013.
  18. Yamamoto, M., Hoang, A-T., Koide, T.: Speed traffic sign recognition algorithm for real-time driving assistant system, Proc. 18th Workshop on Synthesis And System Integration of Mixed Information Technologies (SASIMI 2013), pp. 195-200, 2013.
  19. Zaklouta, F. and Stanciulescu, B.: Segmentation masks for real-time traffic sign recognition using weighted HOG-based trees, Proc. 14th Intl. IEEE Conf. Intelligent Transportation Systems (ITSC), pp. 1954-1959, 2011.
  20. Zaklouta, F., Stanciulescu, B.: Real-time traffic sign recognition in three stages, Journal of Robotics and Autonomous Systems, Vol 62, Issue 1, pp. 16-24, 2014. https://doi.org/10.1016/j.robot.2012.07.019
  21. Ciresan, D., et al.: Multi-column deep neural network for traffic sign classification, Journal of Neural Networks, Vol 32, pp. 333-338, 2012. https://doi.org/10.1016/j.neunet.2012.02.023
  22. Sermanet, P., LeCun, Y.: Traffic sign recognition with multi-scale convolutional networks, Proc. Intl. Joint IEEE Conf. on Neural Networks (IJCNN), pp. 2809-2813, 2011.
  23. Zaklouta, F., Stanciulescu, B., and Hamdoun. O: Traffic sign classification using K-d trees and random forests, Proc. Intl. Joint IEEE Conf. on Neural Networks (IJCNN), pp. 2151-2155, 2011.
  24. Article (CrossRef Link), access at March, 15th, 2015
  25. Article (CrossRef Link), access at June, 10th, 2015.
  26. Hoang, A-T., Yamamoto, M., Koide, T.: Simple yet effective two-stage speed traffic sign recognition for robust vehicle environments, Proc. 30th Intl. Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC 2015), pp.420-423, 2015.