Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Impacts of label quality on performance of steel fatigue crack recognition using deep learning-based image segmentation

  • Hsu, Shun-Hsiang (NCREE-NTUCE Joint Artificial Intelligence Research Center) ;
  • Chang, Ting-Wei (Department of Civil Engineering, National Taiwan University) ;
  • Chang, Chia-Ming (Department of Civil Engineering, National Taiwan University)
  • Received : 2021.04.30
  • Accepted : 2021.07.09
  • Published : 2022.01.25
⟨ Previous Next ⟩

Abstract

Structural health monitoring (SHM) plays a vital role in the maintenance and operation of constructions. In recent years, autonomous inspection has received considerable attention because conventional monitoring methods are inefficient and expensive to some extent. To develop autonomous inspection, a potential approach of crack identification is needed to locate defects. Therefore, this study exploits two deep learning-based segmentation models, DeepLabv3+ and Mask R-CNN, for crack segmentation because these two segmentation models can outperform other similar models on public datasets. Additionally, impacts of label quality on model performance are explored to obtain an empirical guideline on the preparation of image datasets. The influence of image cropping and label refining are also investigated, and different strategies are applied to the dataset, resulting in six alternated datasets. By conducting experiments with these datasets, the highest mean Intersection-over-Union (mIoU), 75%, is achieved by Mask R-CNN. The rise in the percentage of annotations by image cropping improves model performance while the label refining has opposite effects on the two models. As the label refining results in fewer error annotations of cracks, this modification enhances the performance of DeepLabv3+. Instead, the performance of Mask R-CNN decreases because fragmented annotations may mistake an instance as multiple instances. To sum up, both DeepLabv3+ and Mask R-CNN are capable of crack identification, and an empirical guideline on the data preparation is presented to strengthen identification successfulness via image cropping and label refining.

Keywords

Acknowledgement

The authors would like to thank the organizations of the International Project Competition for SHM (IPC-SHM 2020) ANCRiSST, Harbin Institute of Technology (China), and University of Illinois at Urbana-Champaign (USA) for their generosity of providing the invaluable data. The authors also would like to thank the chairs of IPC-SHM 2020 Prof. Hui Li, and Prof. Billie F. Spencer Jr. for their leadership on the competition.

References

  1. Abdellatif, M., Peel, H., Cohn, A.G. and Fuentes, R. (2020), "Combining block-based and pixel-based approaches to improve crack detection and localization", Automat. Constr., 122, 103492. https://doi.org/10.1016/j.autcon.2020.103492
  2. Albrecht, P. and Wright, W. (2000), "Fatigue and fracture of steel bridges", Eur. Struct. Integr. Soc., 26, 211-234. https://doi.org/10.1016/S1566-1369(00)80051-5
  3. Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017), "Segnet: a deep convolutional encoder-decoder architecture for image segmentation", IEEE Transact. Pattern Anal. Mach. Intell., 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615
  4. Breheret, A. (2017), "Pixel Annotation Tool", Retrieved from: https://github.com/abreheret/PixelAnnotationTool
  5. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2017), "Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs", IEEE Transact. Pattern Anal. Mach. Intell., 40(4), 834-848. https://doi.org/10.1109/TPAMI.2017.2699184
  6. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018), "Encoder-decoder with atrous separable convolution for semantic image segmentation", Proceedings of the European Conference on Computer Vision, Munich, Germany, September.
  7. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K. and Fei-Fei, L. (2009), "Imagenet: a large-scale hierarchical image database", IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, June.
  8. Dung, C.V. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028
  9. Farrar, C.R. and Worden, K. (2007), "An introduction to structural health monitoring", Philos. Transact. Royal Soc. A: Mathe. Phys. Eng. Sci., 365(1851), 303-315. https://doi.org/10.1098/rsta.2006.1928
  10. Feng, D. and Feng, M.Q. (2018), "Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection-A review", Eng. Struct., 156, 105-117. https://doi.org/10.1016/j.engstruct.2017.11.018
  11. He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA, June.
  12. He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), "Mask rcnn", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October.
  13. Hsu, S.-H., Chang, T.-W. and Chang, C.-M. (2020), "Concrete Surface Crack Segmentation Based on Deep Learning", In: European Workshop on Structural Health Monitoring (Lecture Notes in Civil Engineering), Vol. 128, pp. 24-34. https://doi.org/10.1007/978-3-030-64908-1_3
  14. Jegou, S., Drozdzal, M., Vazquez, D., Romero, A. and Bengio, Y. (2017), "The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, July.
  15. Kalfarisi, R., Wu, Z.Y. and Soh, K. (2020), "Crack detection and segmentation using deep learning with 3D reality mesh model for quantitative assessment and integrated visualization", J. Comput. Civil Eng., 34(3), 04020010. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000890
  16. Kiefer, J. and Wolfowitz, J. (1952), "Stochastic estimation of the maximum of a regression function", Math. Statist., 23(3), 462-466. https://doi.org/10.1214/aoms/1177729392
  17. Kirk, D. (2007), "NVIDIA CUDA software and GPU parallel computing architecture", Proceedings of the 6th International Symposium on Memory Management, New York, NY, USA, October.
  18. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P. and Zitnick, C.L. (2014), "Microsoft coco: common objects in context", Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, September.
  19. Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S. (2017), "Feature pyramid networks for object detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, July.
  20. Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June.
  21. Meng, Z., Fan, X., Chen, X., Chen, M. and Tong, Y. (2017), "Detecting small signs from large images", IEEE International Conference on Information Reuse and Integration, San Diego, CA, USA, August.
  22. Mohan, A. and Poobal, S. (2018), "Crack detection using image processing: a critical review and analysis", Alexandria Eng. J., 57(2), 787-798. https://doi.org/10.1016/j.aej.2017.01.020
  23. Ozge Unel, F., Ozkalayci, B.O. and Cigla, C. (2019), "The power of tiling for small object detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, June.
  24. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. and Desmaison, A. (2019), "PyTorch: an imperative style, high-performance deep learning library", Adv. Neural Inform. Process. Syst., 32, 8024-8035.
  25. Redmon, J. and Farhadi, A. (2018), "Yolov3: an incremental improvement", arXiv preprint, arXiv: 1804.02767.
  26. Ren, S., He, K., Girshick, R. and Sun, J. (2015), "Faster r-cnn: towards real-time object detection with region proposal networks", IEEE Transact. Pattern Anal. Mach. Intell., 39(6), 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
  27. Song, G., Wang, C. and Wang, B. (2017), "Structural health monitoring (SHM) of civil structures", Appl. Sci., 7(8), 789. https://doi.org/10.3390/app7080789
  28. Song, Q., Wu, Y., Xin, X., Yang, L., Yang, M., Chen, H., Liu, C., Hu, M., Chai, X. and Li, J. (2019), "Real-time tunnel crack analysis system via deep learning", IEEE Access, 7, 64186-64197. https://doi.org/10.1109/ACCESS.2019.2916330
  29. Wada, K. (2016), "labelme: Image Polygonal Annotation with Python", Retrieved from: https://github.com/wkentaro/labelme
  30. Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y. and Girshick, R. (2019), "Detectron2", Retrieved from: https://github.com/facebookresearch/detectron2
  31. Zhou, C. (2020), "Yolact++ better real-time instance segmentation", Ph.D. Dissertation; University of California, Davis, CA, USA.