The Journal of Society for e-Business Studies (한국전자거래학회지)

Society for e-Business Studies (한국전자거래학회)
권호 표지
DOI QR코드

DOI QR Code

A Development of Road Crack Detection System Using Deep Learning-based Segmentation and Object Detection

딥러닝 기반의 분할과 객체탐지를 활용한 도로균열 탐지시스템 개발

  • Ha, Jongwoo (Major in Industrial Data Science & Engineering, Department of Industrial and Data Engineering, Pukyong National University) ;
  • Park, Kyongwon (PaveTec Korea Co., Ltd.) ;
  • Kim, Minsoo (Major in Industrial Data Science & Engineering, Department of Industrial and Data Engineering, Pukyong National University)
  • Received : 2021.01.14
  • Accepted : 2021.02.09
  • Published : 2021.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Many recent studies on deep learning-based road crack detection have shown significantly more improved performances than previous works using algorithm-based conventional approaches. However, many deep learning-based studies are still focused on classifying the types of cracks. The classification of crack types is highly anticipated in that it can improve the crack detection process, which is currently relying on manual intervention. However, it is essential to calculate the severity of the cracks as well as identifying the type of cracks in actual pavement maintenance planning, but studies related to road crack detection have not progressed enough to automated calculation of the severity of cracks. In order to calculate the severity of the crack, the type of crack and the area of the crack in the image must be identified together. This study deals with a method of using Mobilenet-SSD that is deep learning-based object detection techniques to effectively automate the simultaneous detection of crack types and crack areas. To improve the accuracy of object-detection for road cracks, several experiments were conducted to combine the U-Net for automatic segmentation of input image and object-detection model, and the results were summarized. As a result, image masking with U-Net is able to maximize object-detection performance with 0.9315 mAP value. While referring the results of this study, it is expected that the automation of the crack detection functionality on pave management system can be further enhanced.

최근 도로균열 탐지에 대한 많은 연구에서 딥러닝 기반의 접근법을 활용하면서 과거 알고리즘 기반의 접근법을 활용한 연구들보다 높은 성능과 성과를 보이고 있다. 그러나 딥러닝 기반의 많은 연구가 여전히 균열의 유형을 분류하는 것에 집중되어 있다. 균열 유형의 분류는 현재 수작업에 의존하고 있는 균열탐지 프로세스를 획기적으로 개선해 줄 수 있다는 점에서 상당한 기대를 받고 있다. 그러나 실제 도로의 유지보수 작업에 있어서는 균열의 유형뿐만 아니라 균열의 심각도에 관한 판단이 필수적이지만, 아직까지 도로균열 탐지와 관련된 연구들이 균열의 심각도에 대한 자동화된 산출까지 진전되지 못하고 있다. 균열의 심각도를 산출하기 위해서는 균열의 유형과 이미지 속 균열의 부위가 함께 파악되어야 한다. 본 연구에서는 균열 유형과 균열 부위의 동시적 탐지를 효과적으로 자동화하기 위해 딥러닝 기반의 객체탐지 모델인 Mobilenet-SSD를 활용하는 방법을 다루고 있다. 균열탐지의 정확도를 개선하기 위해 U-Net을 활용해 입력 이미지를 자동 분할하고, 이를 객체탐지 기법과 결합하기 위한 여러 실험을 진행하여 그 결과를 정리하였다. 결과적으로 U-Net을 활용한 이미지 의 자동 마스킹을 통해 객체탐지의 성능을 mAP 값이 0.9315가 되도록 향상시킬 수 있었다. 본 연구의 결과를 참고하여 도로포장 관리시스템의 구현에 균열탐지 기능의 자동화가 더욱 진전될 수 있다고 기대된다.

Keywords

References

  1. Ashraf, S., Hegazy, I., and Elarif, T. L., "Algorithm for Automatic Crack Analysis and Severity Identification," 2019 IEEE Ninth International Conference on Intelligent Computing and Information Systems, pp. 74-79, 2019.
  2. Cubero-Fernandez, A., Rodriguez-Lozano, F. J., Villatoro, R., Olivares, J., and Palomares, J. M., "Efficient pavement crack detection and classification," EURASIP Journal on Image and Video Processing, Vol. 39, pp. 1-11, 2017.
  3. Feng, C., Liu, M. Y., Kao, C. C., and Lee, T. Y., "Deep active learning for civil infrastructure defect detection and classification," Computing in Civil Engineering, pp. 298-306, 2017.
  4. Gopalakrishnan, K., "Deep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review," Data, Vol. 3, No. 3, pp. 1-19, 2018. https://doi.org/10.3390/data3030028
  5. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., and Adam, H., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704. 04861, 2017.
  6. https://github.com/tensorflow/models/tree/master/research/object_detection.
  7. Jo, H., Kim, D., Pak, K. W., and Kim, M., "Road damage detection over road scanner images using deep convolutional neural network," ICIC Express Letters, Vol. 14, No. 10, pp. 1001-1008, 2020. https://doi.org/10.3144/expresspolymlett.2020.81
  8. Kim, M., Ryu, J., Cha, D., and Sim, M. K., "Stock Price Prediction Using Sentiment Analysis: from "Stock Discussion Room in Naver," The Journal of Society for e-Business Studies, Vol. 25, No. 4, pp. 61-75, 2020. https://doi.org/10.7838/JSEBS.2020.25.4.061
  9. Kirillov, A., Girshick, R., He, K., and Dollar, P., "Panoptic Feature Pyramid Networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6399-6408, 2019.
  10. Koch, C. and Brilakis, I., "Pothole detection in asphalt pavement images," Advanced Engineering Informatics, Vol. 25, No. 3, pp. 507-515, 2011. https://doi.org/10.1016/j.aei.2011.01.002
  11. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., and Berg, A. C., "Ssd: Single shot multibox detector," European Conference on Computer Vision, Springer, Cham, pp. 21-37, 2016.
  12. Naddaf-Sh, M., Hosseini, S., Zhang, J., Brake, N. A., and Zargarzadeh, H., "Realtime road crack mapping using an optimized convolutional neural network," Complexity, pp. 1-17, 2019.
  13. Pauly, L., Hogg, D., Fuentes, R., and Peel, H., "Deeper networks for pavement crack detection," Proceedings of the 34th International Symposium on Automation and Robotics in Construction(ISARC), IAARC, pp. 479-485, 2017.
  14. Rababaah, H., Vrajitoru, D., and Wolfer, J., "Asphalt pavement crack classification: a comparison of GA, MLP, and SOM," Proceedings of Genetic and Evolutionary Computation Conference, Late-Breaking Paper, 2005.
  15. Ragnoli, A., De Blasiis, M. R., and Benedetto, A. D., "Pavement Distress Detection Methods: A Review," MDPI Infrastructure, Vol. 3, No. 58, pp. 1-19, 2018. https://doi.org/10.3390/infrastructures3010001
  16. Ronneberger, O., Fischer, P., and Brox, T., "U-net: Convolutional networks for biomedical image segmentation," International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, pp. 234-241, 2015.
  17. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. C., "Mobilenetv2: Inverted residuals and linear bottlenecks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
  18. Sorncharean, S. and Phiphobmongkol, S., "Crack detection on asphalt surface image using enhanced grid cell analysis," 4th IEEE International Symposium on Electronic Design, Test and Applications, pp. 49-54, 2008.
  19. Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., and Ling, H., "Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection," IEEE Transactions on Intelligent Transportation Systems, Vol. 21, No. 4, pp. 1525-1535, 2020. https://doi.org/10.1109/tits.2019.2910595
  20. Yoo, W., Seo, J., Kim, D., and Kim, K., "Machine Scheduling Models Based on Reinforcement Learning for Minimizing Due Date Violation and Setup Chang," The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 19-33, 2019.
  21. Yoo, W., Seo, J., Lee, D., Kim, D., and Kim, K., "Scheduling Generation Model on Parallel Machines with Due Date and Setup Cost Based on Deep Learning," The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 99-110, 2019.
  22. Zhang, A., Wang, K. C., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J. Q., and Chen, C., "Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep‐learning network," Computer-Aided Civil and Infrastructure Engineering, Vol. 32, No. 10, pp. 805-819, 2017. https://doi.org/10.1111/mice.12297
  23. Zhang, L., Yang, F., Zhang, Y. D., and Zhu, Y. J., "Road crack detection using deep convolutional neural network," Proceedings of the 2016 IEEE International Conference on Image Processing(ICIP), Phoenix, AZ, USA, 25-28 September 2016, pp. 3708-3712, 2016.