Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Smartphone-based structural crack detection using pruned fully convolutional networks and edge computing

  • Ye, X.W. (Department of Civil Engineering, Zhejiang University) ;
  • Li, Z.X. (Department of Civil Engineering, Zhejiang University) ;
  • Jin, T. (Department of Civil Engineering, Zhejiang University)
  • Received : 2021.04.15
  • Accepted : 2021.08.06
  • Published : 2022.01.25
⟨ Previous Next ⟩

Abstract

In recent years, the industry and research communities have focused on developing autonomous crack inspection approaches, which mainly include image acquisition and crack detection. In these approaches, mobile devices such as cameras, drones or smartphones are utilized as sensing platforms to acquire structural images, and the deep learning (DL)-based methods are being developed as important crack detection approaches. However, the process of image acquisition and collection is time-consuming, which delays the inspection. Also, the present mobile devices such as smartphones can be not only a sensing platform but also a computing platform that can be embedded with deep neural networks (DNNs) to conduct on-site crack detection. Due to the limited computing resources of mobile devices, the size of the DNNs should be reduced to improve the computational efficiency. In this study, an architecture called pruned crack recognition network (PCR-Net) was developed for the detection of structural cracks. A dataset containing 11000 images was established based on the raw images from bridge inspections. A pruning method was introduced to reduce the size of the base architecture for the optimization of the model size. Comparative studies were conducted with image processing techniques (IPTs) and other DNNs for the evaluation of the performance of the proposed PCR-Net. Furthermore, a modularly designed framework that integrated the PCR-Net was developed to realize a DL-based crack detection application for smartphones. Finally, on-site crack detection experiments were carried out to validate the performance of the developed system of smartphone-based detection of structural cracks.

Keywords

Acknowledgement

The work described in this paper was jointly supported by the National Natural Science Foundation of China (Grant Nos. 52178306, 51822810 and 51778574), and the Zhejiang Provincial Natural Science Foundation of China (Grant No. LR19E080002). The authors would like to thank the organizations of the International Project Competition for SHM (IPC-SHM 2020) ANCRiSST, Harbin Institute of Technology (China), and the University of Illinois at Urbana-Champaign (USA) for their generously providing the invaluable data from actual structures. The authors also would like to thank the chairs of IPC-SHM 2020 Prof. Hui Li, and Prof. Billie F. Spencer Jr. for their leadership in the competition.

References

  1. Abdel-Qader, I., Abudayyeh, O. and Kelly, M.E. (2003), "Analysis of edge-detection techniques for crack identification in bridges", J. Comput. Civil Eng., 17(4), 255-263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)
  2. Alipour, M., Harris, D.K. and Miller, G.R. (2019), "Robust pixel-level crack detection using deep fully convolutional neural networks", J. Comput. Civil Eng., 33(6), 04019040. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000854
  3. Bao, Y.Q., Tang, Z.Y. and Li, H. (2019), "Compressive-sensing data reconstruction for structural health monitoring: a machine-learning approach", Struct. Health Monitor., 19(1), 293-304. https://doi.org/10.1177/1475921719844039
  4. Fujita, Y. and Hamamoto, Y. (2011), "A robust automatic crack detection method from noisy concrete surfaces", Mach. Vision Appl., 22(2), 245-254. https://doi.org/10.1007/s00138-009-0244-5
  5. Gresil, M., Yu, L., Shen, Y. and Giurgiutiu, V. (2013), "Predictive model of fatigue crack detection in thick bridge steel structures with piezoelectric wafer active sensors", Smart Struct. Syst., Int. J., 12(2), 97-119. https://doi.org/10.12989/sss.2013.12.2.097
  6. Hakim, S.J.S. and Razak, H.A. (2014), "Modal parameters based structural damage detection using artificial neural networks - a review", Smart Struct. Syst., Int. J., 14(2), 159-189. https://doi.org/10.12989/sss.2014.14.2.159
  7. Jang, J., Shin, M., Lim, S., Park, J. and Paik, J. (2019), "Intelligent image-based railway inspection system using deep learning-based object detection and weber contrast-based image comparison", Sensors, 19(21), 4738. https://doi.org/10.3390/s19214738
  8. Kingma, D.P. and Ba, J.L. (2015), "Adam: a method for stochastic optimization", Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA. (CD-ROM)
  9. Li, S.Y. and Zhao, X.F. (2019), "Image-based concrete crack detection using convolutional neural network and exhaustive search technique", Adv. Civil Eng., 2019. https://doi.org/10.1155/2019/6520620
  10. Li, S.Y. and Zhao, X.F. (2020), "Automatic crack detection and measurement of concrete structure using convolutional encoder-decoder network", IEEE Access, 8, 134602-134618. https://doi.org/10.1109/ACCESS.2020.3011106
  11. Li, G., Ma, B., He, S.H., Ren, X.L. and Liu, Q.W. (2020), "Automatic tunnel crack detection based on u-net and a convolutional neural network with alternately updated clique", Sensors, 20(3), 717. https://doi.org/10.3390/s20030717
  12. Liu, Q. (2019), U-Net Implementation in PyTorch. Retrieved from https://github.com/Qiuyan918/Unet_Implementation_PyTorch/b lob/master/Unet_Implementation_PyTorch.ipynb
  13. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S. and Zhang, C. (2017), "Learning efficient convolutional networks through network slimming", Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, Italy. (CD-ROM) https://doi.org/10.1109/ICCV.2017.298
  14. Mondal, T.G. and Jahanshahi, M.R. (2020), "Autonomous vision-based damage chronology for spatiotemporal condition assessment of civil infrastructure using unmanned aerial vehicle", Smart Struct. Syst., Int. J., 25(6), 733-749. https://doi.org/10.12989/sss.2020.25.6.733
  15. Ni, Y.Q., Ye, X.W. and Ko, J.M. (2010), "Monitoring-based fatigue reliability assessment of steel bridges: analytical model and application", J. Struct. Eng., 136(12), 1563-1573. https://doi.org/10.1061/(ASCE)ST.1943-541X.0000250
  16. Ni, Y.Q., Ye, X.W. and Ko, J.M. (2012), "Modeling of stress spectrum using long-term monitoring data and finite mixture distributions", J. Eng. Mech., 138(2), 175-183. https://doi.org/10.1061/(ASCE)EM.1943-7889.0000313
  17. Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: convolutional networks for biomedical image segmentation", Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Intervention, Munich, Germany. (CD-ROM) https://doi.org/10.1007/978-3-319-24574-4_28
  18. Ryu, E., Kang, J., Lee, J., Shin, Y. and Kim, H. (2020), "Automated detection of surface cracks and numerical correlation with thermal-structural behaviors of fire damaged concrete beams", Int. J. Concrete Struct. Mater., 14(1), 12. https://doi.org/10.1186/s40069-019-0387-3
  19. Spencer Jr., B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  20. Tang, Z.Y., Chen, Z.C., Bao, Y.Q. and Li, H. (2019), "Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring", Struct. Control. Health Monitor., 26(1), e2296. https://doi.org/10.1002/stc.2296
  21. Xu, Y., Wei, S.Y., Bao, Y.Q. and Li, H. (2019), "Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network", Struct. Control. Health Monitor., 26(3), e2313. https://doi.org/10.1002/stc.2313
  22. Ye, X.W., Ni, Y.Q., Wong, K.Y. and Ko, J.M. (2012), "Statistical analysis of stress spectra for fatigue life assessment of steel bridges with structural health monitoring data", Eng. Struct., 45, 166-176. https://doi.org/10.1016/j.engstruct.2012.06.016
  23. Ye, X.W., Ni, Y.Q., Wai, T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013), "A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification", Smart Struct. Syst., Int. J., 12(3-4), 363-379. https://doi.org/10.12989/sss.2013.12.3_4.363
  24. Ye, X.W., Jin, T. and Yun, C.B. (2019a), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst., Int. J., 24(5), 567-585. https://doi.org/10.12989/sss.2019.24.5.567
  25. Ye, X.W., Jin, T. and Chen, P.Y. (2019b), "Structural crack detection using deep learning-based fully convolutional networks", Adv. Struct. Eng., 22(16), 3412-3419. https://doi.org/10.1177/1369433219836292
  26. Ye, X.W., Jin, T., Ang, P.P., Bian, X.C. and Chen, Y.M. (2021), "Computer vision-based monitoring of the 3-D structural deformation of an ancient structure induced by shield tunneling construction", Struct. Control. Health Monitor., 28(4), e2702. https://doi.org/10.1002/stc.2702