Structural Engineering and Mechanics

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Research on damage detection and assessment of civil engineering structures based on DeepLabV3+ deep learning model

  • Chengyan Song (College of Civil Engineering and Transportation, Hebei University of Technology)
  • Received : 2024.04.16
  • Accepted : 2024.08.06
  • Published : 2024.09.10
⟨ Previous Next ⟩

Abstract

At present, the traditional concrete surface inspection methods based on artificial vision have the problems of high cost and insecurity, while the computer vision methods rely on artificial selection features in the case of sensitive environmental changes and difficult promotion. In order to solve these problems, this paper introduces deep learning technology in the field of computer vision to achieve automatic feature extraction of structural damage, with excellent detection speed and strong generalization ability. The main contents of this study are as follows: (1) A method based on DeepLabV3+ convolutional neural network model is proposed for surface detection of post-earthquake structural damage, including surface damage such as concrete cracks, spaling and exposed steel bars. The key semantic information is extracted by different backbone networks, and the data sets containing various surface damage are trained, tested and evaluated. The intersection ratios of 54.4%, 44.2%, and 89.9% in the test set demonstrate the network's capability to accurately identify different types of structural surface damages in pixel-level segmentation, highlighting its effectiveness in varied testing scenarios. (2) A semantic segmentation model based on DeepLabV3+ convolutional neural network is proposed for the detection and evaluation of post-earthquake structural components. Using a dataset that includes building structural components and their damage degrees for training, testing, and evaluation, semantic segmentation detection accuracies were recorded at 98.5% and 56.9%. To provide a comprehensive assessment that considers both false positives and false negatives, the Mean Intersection over Union (Mean IoU) was employed as the primary evaluation metric. This choice ensures that the network's performance in detecting and evaluating pixel-level damage in post-earthquake structural components is evaluated uniformly across all experiments. By incorporating deep learning technology, this study not only offers an innovative solution for accurately identifying post-earthquake damage in civil engineering structures but also contributes significantly to empirical research in automated detection and evaluation within the field of structural health monitoring.

Keywords

Acknowledgement

The authors wish to express their sincere gratitude to the contributors who have directly or indirectly supported our research. Special thanks to the academic and technical staff of (Your Institution's Name) for their invaluable assistance and insights throughout this study. We also extend our appreciation to the reviewers for their constructive feedback that significantly enhanced the quality of this manuscript. Our acknowledgment would not be complete without thanking (Any Specific Grant or Funding Information, if applicable), which provided the financial support necessary to carry out this research. Lastly, we are grateful to all the colleagues and team members who contributed their time and expertise to the success of this project.

References

  1. Anand, S., Gupta, S., Darbari, V. and Kohli, S. (2018), "Crackpot: Autonomous road crack and pothole detection", 2018 Digital Image Computing: Techniques and Applications (DICTA), 1-6.
  2. Angeletti, F., Gasbarri, P., Panella, M. and Rosato, A. (2023), "Multi-damage detection in composite space structures via deep learning", Sensor., 23(17), 7515. https://doi.org/10.3390/s23176245.
  3. Arnold, T., De Biasio, M., Fritz, A. and Leitner, R. (2013), "UAV-based measurement of vegetation indices for environmental monitoring", 2013 Seventh International Conference on Sensing Technology (ICST), 704-707.
  4. Asghari, A., Ghodrati Amiri, G., Darvishan, E. and Asghari, A. (2024), "A novel approach for structural damage detection using multi-headed stacked deep ensemble learning", J. Vib. Eng. Technol., 12(3), 4209-4224. https://doi.org/10.1007/s42417-023-00769-1.
  5. Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learningbased crack damage detection using convolutional neural networks", Comput.-Aid. Civil Infrastr. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263.
  6. Chamangard, M., Ghodrati Amiri, G., Darvishan, E. and Rastin, Z. (2022), "Transfer learning for CNN-based damage detection in civil structures with insufficient data", Shock Vib., 2022(1), 3635116. https://doi.org/10.1155/2022/1816835.
  7. Chen, B., Fan, M., Li, K., Gao, Y., Wang, Y., Chen, Y., ... & Sun, J. (2024), "The PFILSTM model: A crack recognition method based on pyramid features and memory mechanisms", Front. Mater., 10, 1347176. https://doi.org/10.3969/j.issn.1001-7372.2023.02.001.
  8. Chen, Z., Wang, C., Wu, J., Deng, C. and Wang, Y. (2023), "Deep convolutional transfer learning-based structural damage detection with domain adaptation", Appl. Intel., 53(5), 5085-5099. https://doi.org/10.1007/s10489-021-02779-2.
  9. Choi, W. and Cha, Y.J. (2019), "SDDNet: Real-time crack segmentation", IEEE Trans. Indus. Electr., 67(9), 8016-8025. https://doi.org/10.1016/j.autcon.2020.103254.
  10. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., ... & Schiele, B. (2016), "The cityscapes dataset for semantic urban scene understanding", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3213-3223. https://doi.org/10.1109/CVPR.2016.182.
  11. Dunphy, K., Fekri, M.N., Grolinger, K. and Sadhu, A. (2022), "Data augmentation for deep-learning-based multiclass structural damage detection using limited information", Sensor., 22(16), 6193. https://doi.org/10.3390/s22166193.
  12. Eves, A.J., Stewart, T.P., Gay, A.P., Kemp, A.I., Easey, M., Angel, R., ... & Pearce, D.A. (2009), "Developing unmanned aerial vehicles for local and flexible environmental and agricultural monitoring", New Dimensions in Earth Observation. Remote Sensing and Photogrammetry Society Conference, Leicester, UK.
  13. Guo, X., Wang, N. and Li, Y. (2023), "Enhancing pavement maintenance: A deep learning model for accurate prediction and early detection of pavement structural damage", Constr. Build. Mater., 409, 133970. https://doi.org/10.1016/j.conbuildmat.2023.128521.
  14. Haciefendioglu, K., Ayas, S., Basaga, H.B., Togan, V., Mostofi, F. and Can, A. (2022), "Wood construction damage detection and localization using deep convolutional neural network with transfer learning", Eur. J. Wood Wood Prod., 80(4), 791-804. https://doi.org/10.1007/s00107-022-01781-4.
  15. Haciefendioglu, K., Basaga, H.B., Kartal, M.E. and Bulut, M.C. (2022), "Automatic damage detection on traditional wooden structures with deep learning-based image classification method", Drvna Industrija, 73(2), 163-176. https://doi.org/10.5552/drvind.2022.2158.
  16. Han, Q., Zhao, N. and Xu, J. (2021), "Recognition and location of steel structure surface corrosion based on unmanned aerial vehicle images", J. Civil Struct. Hlth. Monit., 11(5), 1375-1392. https://doi.org/10.1007/s13349-021-00464-3.
  17. Han, X., Zhao, Z., Chen, L., Hu, X., Tian, Y., Zhai, C., ... & Huang, X. (2022), "Structural damage-causing concrete cracking detection based on a deep-learning method", Constr. Build. Mater., 337, 127562. https://doi.org/10.1016/j.conbuildmat.2022.128325.
  18. Hoskere, V., Narazaki, Y., Hoang, T. and Spencer Jr, B. (2018), "Vision-based structural inspection using multiscale deep convolutional neural networks", arXiv preprint arXiv:1805.01055. https://doi.org/10.48550/arXiv.1805.01055.
  19. Iannelli, P., Angeletti, F., Gasbarri, P., Panella, M. and Rosato, A. (2022), "Deep learning-based structural health monitoring for damage detection on a large space antenna", Acta Astronautica, 193, 635-643. https://doi.org/10.1016/j.actaastro.2021.10.042.
  20. Jang, K., An, Y.K., Kim, B. and Cho, S. (2021), "Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot", Comput.-Aid. Civil Infrastr. Eng., 36(1), 14-29. https://doi.org/10.1111/mice.12552.
  21. Kang, D. and Cha, Y.J. (2018), "Damage detection with an autonomous UAV using deep learning", Sensor. Smart Struct. Technol. Civil, Mech. Aerosp. Syst., 10598, 7-14. https://doi.org/10.1117/12.2318766.
  22. Kim, I.H., Jeon, H., Baek, S.C., Hong, W.H. and Jung, H.J. (2018), "Application of crack identification techniques for an aging concrete bridge inspection using an unmanned aerial vehicle", Sensor., 18(6), 1881. https://doi.org/10.3390/s18060163.
  23. Lee, Y., Kim, H., Min, S. and Yoon, H. (2023), "Structural damage detection using deep learning and FE model updating techniques", Scientif. Report., 13(1), 18694. https://doi.org/10.1038/s41598-023-18694-0.
  24. Lingxin, Z., Junkai, S. and Baijie, Z. (2022), "A review of the research and application of deep learning-based computer vision in structural damage detection", Earthq. Eng. Eng. Vib., 21(1), 1-21. https://doi.org/10.1007/s11803-022-2787-0.
  25. Liu, C., Xu, X., Wu, J., Zhu, H. and Wang, C. (2023), "Deep transfer learning-based damage detection of composite structures by fusing monitoring data with physical mechanism", Eng. Appl. Artif. Intel., 123, 106245. https://doi.org/10.1016/j.engappai.2022.104545.
  26. Lomazzi, L., Giglio, M. and Cadini, F. (2023), "Towards a deep learning-based unified approach for structural damage detection, localisation and quantification", Eng. Appl. Artif. Intel., 121, 106003. https://doi.org/10.1016/j.engappai.2023.105324.
  27. Ma, J., Hu, M., Yang, Z., Yang, H., Ma, S., Xu, H., ... & Wu, Z. (2023), "An efficient lightweight deep-learning approach for guided Lamb wave-based damage detection in composite structures", Appl. Sci., 13(8), 5022. https://doi.org/10.3390/app13083946.
  28. Mousavi, Z., Varahram, S., Ettefagh, M.M., Sadeghi, M.H., Feng, W.Q. and Bayat, M. (2024), "A digital twin-based framework for damage detection of a floating wind turbine structure under various loading conditions based on deep learning approach", Ocean Eng., 292, 116563. https://doi.org/10.1016/j.oceaneng.2024.116563.
  29. Nguyen, T.Q. (2022), "Damage detection in beam structures using bayesian deep learning and balancing composite motion optimization", Struct., 39, 98-114. https://doi.org/10.1016/j.istruc.2021.12.012.
  30. Nguyen, T.Q., Nguyen, T.A. and Nguyen, T.T. (2023), "Detecting structural deterioration: Investigating changes in power spectral density using deep learning on damaged steel beam structures", Int. J. Steel Struct., 23(4), 929-944. https://doi.org/10.1007/s13296-022-00766-6.
  31. Park, G., Farrar, C.R., Di Scalea, F.L. and Coccia, S. (2006), "Performance assessment and validation of piezoelectric activesensors in structural health monitoring", Smart Mater. Struct., 15(6), 1673. https://doi.org/10.1088/0964-1726/15/6/033.
  32. Protopapadakis, E., Voulodimos, A., Doulamis, A., Doulamis, N. and Stathaki, T. (2019), "Automatic crack detection for tunnel inspection using deep learning and heuristic image postprocessing", Appl. Intel., 49, 2793-2806. https://doi.org/10.1007/s10489-019-01439-8.
  33. Qiu, X., Han, F. and Zhao,W. (2023), "Real-time intelligent crack detection algorithm for underwater robot", J. Harbin Eng. Univ., 53(9), 1-8. https://doi.org/10.11990eu.202203025. https://doi.org/10.11990eu.202203025
  34. Ren, X.Y., Wang, J., Liang, Y.J., Ma, L.H. and Zhou, W. (2024), "Acoustic emission detection of filament wound CFRP composite structure damage based on Mel spectrogram and deep learning", Thin Wall. Struct., 198, 111683. https://doi.org/10.1016/j.tws.2024.111683.
  35. Satpathy, R.P.K., Kumar, K., Hirwani, C.K., Kumar, V., Kumar, E.K. and Panda, S.K. (2023), "Computational deep learning algorithm (vision/frequency response)-based damage detection in engineering structure", Acta Mechanica, 234(11), 5919-5935. https://doi.org/10.1007/s00707-023-03291-7.
  36. Shi, M. and Gao, J. (2022), "Research on pavement crack detection with improved U-Net algorithm", Auto. Instrumen., 37(10), 52-55+67.
  37. Spencer Jr, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Eng., 5(2), 199-222. https://doi.org/10.1016/j.eng.2019.01.019.
  38. Tabatabaeian, A., Jerkovic, B., Harrison, P., Marchiori, E. and Fotouhi, M. (2023), "Barely visible impact damage detection in composite structures using deep learning networks with varying complexities", Compos. Part B: Eng., 264, 110907. https://doi.org/10.1016/j.compositesb.2023.110243.
  39. Wang, P., Xiao, J., Kawaguchi, K.I. and Wang, L. (2022), "Automatic ceiling damage detection in large-span structures based on computer vision and deep learning", Sustain., 14(6), 3275. https://doi.org/10.3390/su14063275.
  40. Xu, X. and Liu, C. (2024), "Physics-guided deep learning for damage detection in CFRP composite structures", Compos. Struct., 331, 117889. https://doi.org/10.1016/j.compstruct.2024.117889.
  41. Yamada, T., Chida, H. and Takahashi, N. (2023), "A deep learning-based crack damage detection in exterior finish of timber houses in Japan for automated damage diagnostic system", JPN Arch. Rev., 6(1), e12362. https://doi.org/10.1002/2475-8876.12435.
  42. Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X. and Ling, H. (2019), "Feature pyramid and hierarchical boosting network for pavement crack detection", IEEE Trans. Intel. Transp. Syst., 21(4), 1525-1535. https://doi.org/10.1109/TITS.2018.2870871.
  43. Zhang, J., Zhang, J., Teng, S., Chen, G. and Teng, Z. (2022), "Structural damage detection based on vibration signal fusion and deep learning", J. Vib. Eng. Technol., 10(4), 1205-1220. https://doi.org/10.1007/s42417-021-00395-5.
  44. Zhang, Y., Li, X. and Wang, J. (2023), "Deep learning-based structural health monitoring: A comprehensive review", J. Struct. Eng., 149(2), 04023001. https://doi.org/10.1061/(ASCE)ST.1943-541X.0003185.
  45. Zhou, Y., Zhang, H., Fang, Y. and Zhu, Y. (2021), "STRNet: Efficient attention-based deep encoder-decoder for automatic crack segmentation", Auto. Constr., 122, 103486. https://doi.org/10.1016/j.autcon.2021.103486.