Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

Detection of Zebra-crossing Areas Based on Deep Learning with Combination of SegNet and ResNet

SegNet과 ResNet을 조합한 딥러닝에 기반한 횡단보도 영역 검출

  • Liang, Han (Dept. of Civil Engineering, Kyungpook National University) ;
  • Seo, Suyoung (Dept. of Civil Engineering, Kyungpook National University)
  • Received : 2021.05.12
  • Accepted : 2021.06.21
  • Published : 2021.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a method to detect zebra-crossing using deep learning which combines SegNet and ResNet. For the blind, a safe crossing system is important to know exactly where the zebra-crossings are. Zebra-crossing detection by deep learning can be a good solution to this problem and robotic vision-based assistive technologies sprung up over the past few years, which focused on specific scene objects using monocular detectors. These traditional methods have achieved significant results with relatively long processing times, and enhanced the zebra-crossing perception to a large extent. However, running all detectors jointly incurs a long latency and becomes computationally prohibitive on wearable embedded systems. In this paper, we propose a model for fast and stable segmentation of zebra-crossing from captured images. The model is improved based on a combination of SegNet and ResNet and consists of three steps. First, the input image is subsampled to extract image features and the convolutional neural network of ResNet is modified to make it the new encoder. Second, through the SegNet original up-sampling network, the abstract features are restored to the original image size. Finally, the method classifies all pixels and calculates the accuracy of each pixel. The experimental results prove the efficiency of the modified semantic segmentation algorithm with a relatively high computing speed.

본 논문은 SegNet과 ResNet을 조합한 딥러닝을 이용하여 횡단보도를 검출하는 방법을 제안한다. 시각 장애인의 경우 횡단보도가 어디에 있는지 정확히 아는 게 안전한 교통 시스템에서 중요하다. 딥러닝에 의한 횡단보도 검출은 이 문제에 대한 좋은 해결책이 될 수 있다. 로봇 시각 기반 보조 기술은 지난 몇년 동안 카메라를 사용하는 특정 장면에 초점을 두고 제안되어 왔다. 이러한 전통적인 방법은 비교적 긴 처리 시간으로 의미있는 결과를 얻었으며 횡단보도 인식을 크게 향상시켰다. 그러나 전통적인 방법은 지연 시간이 길고 웨어러블 장비에서 실시간을 만족시킬 수 없다. 본 연구에서 제안하는 방법은 취득한 영상에서 횡단보도를 빠르고 안정적으로 검출하기 위한 모델을 제안한다. 모델은 SegNet과 ResNet을 기반으로 개선되었으며 3단계로 구성된다. 첫째, 입력 영상을 서브샘플링하여 이미지 특징을 추출하고 ResNet의 컨벌루션 신경망을 수정하여 새로운 인코더로 만든다. 둘째, 디코딩 과정에서 업샘플링 네트워크를 통해 특징맵을 원영상 크기로 복원한다. 셋째, 모든 픽셀을 분류하고 각 픽셀의 정확도를 계산한다. 이 실험의 결과를 통하여 수정된 시맨틱 분할 알고리즘의 적격한 정확성을 검증하는 동시에 결과 출력 속도가 비교적 빠른 것으로 파악되었다.

Keywords

Acknowledgement

이 논문은 2016년도 정부(교육부)의 재원으로 한국연구재단의 지원을 받아 수행된 기초연구사업임(No. 2016R1D1A1B02011625)

References

  1. Badrinarayanan, V., Kendall, A., and Cipolla, R. (2017), SegNet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615
  2. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2018), DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, No. 4, pp. 834-848. https://doi.org/10.1109/TPAMI.2017.2699184
  3. Fernandes, A.O., Moreira, L.F.E., and Mata, J.M. (2011), Machine vision applications and development aspects, 2011 9th IEEE International Conference on Control and Automation (ICCA). 19-21 December, Santiago, Chile, pp. 1274-1278.
  4. He, K., Zhang, X., Ren, S., and Sun, J. (2016), Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June, Las Vegas, NV, USA, pp. 770-778.
  5. Long, J., Shelhamer, E., and Darrell, T. (2015), Fully convolutional networks for semantic segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 15 October, Los Alamitos, CA, USA, pp. 3431-3440.
  6. Mascetti, S., Ahmetovic, D., Gerino, A., and Bernareggi, C. (2016), ZebraRecognizer: Pedestrian crossing recognition for people with visual impairment or blindness, Pattern Recognition, Vol. 60, pp. 405-419. https://doi.org/10.1016/j.patcog.2016.05.002
  7. Roters, J., Jiang, X., and Rothaus, K. (2011), Recognition of traffic lights in live video streams on mobile devices, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, No. 10, pp. 1497-1511. https://doi.org/10.1109/TCSVT.2011.2163452
  8. Taie, M.A., NasrEldin, K.M., and ElHelw, M. (2013), ITS navigation and live timetables for the blind based on RFID robotic localization algorithms and ZigBee broadcasting, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), 12-14 December, Shenzhen, China, pp. 139-144.
  9. Yang, M., Shi, H., Chen, Y., Ran, X., and Xu, C. (2016), Zebra-crossing automatic recognition and early warning for intelligent driving, International Journal of Signal Processing, Image Processing and Pattern Recognition, Vol. 9, No. 8, pp. 263-274. https://doi.org/10.14257/ijsip.2016.9.8.23
  10. Zhai, Y., Cui, G., Gu, Q., and Kong, L. (2015), Crosswalk detection based on MSER and ERANSAC, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, 15-18 September, Gran Canaria, Spain, pp. 2770-2775.
  11. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017), Pyramid scene parsing network, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July, Honolulu, HI, USA, pp. 6230-6239.
  12. Zhong, J., Feng, W., Lei, Q., Le, S., Wei, X., Wang, Y., and Wang, W. (2020), Improved U-net for zebra-crossing image segmentation, 2020 IEEE 6th International Conference on Computer and Communications (ICCC), 12 February, Chengdu, China, pp. 388-393.