Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Coating defect classification method for steel structures with vision-thermography imaging and zero-shot learning

  • Jun Lee (Department of Civil Engineering, Korean Advanced Institute for Science and Technology) ;
  • Kiyoung Kim (Department of Civil Engineering, Korean Advanced Institute for Science and Technology) ;
  • Hyeonjin Kim (Department of Civil Engineering, Korean Advanced Institute for Science and Technology) ;
  • Hoon Sohn (Department of Civil Engineering, Korean Advanced Institute for Science and Technology)
  • Received : 2023.10.31
  • Accepted : 2023.12.10
  • Published : 2024.01.25
⟨ Previous Next ⟩

Abstract

This paper proposes a fusion imaging-based coating-defect classification method for steel structures that uses zero-shot learning. In the proposed method, a halogen lamp generates heat energy on the coating surface of a steel structure, and the resulting heat responses are measured by an infrared (IR) camera, while photos of the coating surface are captured by a charge-coupled device (CCD) camera. The measured heat responses and visual images are then analyzed using zero-shot learning to classify the coating defects, and the estimated coating defects are visualized throughout the inspection surface of the steel structure. In contrast to older approaches to coating-defect classification that relied on visual inspection and were limited to surface defects, and older artificial neural network (ANN)-based methods that required large amounts of data for training and validation, the proposed method accurately classifies both internal and external defects and can classify coating defects for unobserved classes that are not included in the training. Additionally, the proposed model easily learns about additional classifying conditions, making it simple to add classes for problems of interest and field application. Based on the results of validation via field testing, the defect-type classification performance is improved 22.7% of accuracy by fusing visual and thermal imaging compared to using only a visual dataset. Furthermore, the classification accuracy of the proposed method on a test dataset with only trained classes is validated to be 100%. With word-embedding vectors for the labels of untrained classes, the classification accuracy of the proposed method is 86.4%.

Keywords

Acknowledgement

This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) [Grant Number 2019R1A3B3067987].

References

  1. A.A.o.S.a.H.T. Officials (Ed.) (2019), Standard Practice for Evaluation of Coating Systems for Structural Steel.
  2. Ali, R. and Cha, Y.J. (2019), "Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer", Constr. Build. Mater., 226, 376-387. https://doi.org/10.1016/j.conbuildmat.2019.07.293
  3. Boller, C., Starke, P., Dobmann, G., Kuo, C.M. and Kuo, C.H. (2015), "Approaching the assessment of ageing bridge infrastructure", Smart Struct. Syst., Int. J., 15(3), 593-608. https://doi.org/10.12989/sss.2015.15.3.593
  4. Center, N.L.I. (2021), Detailed guidelines for safety and maintenance of facilities.
  5. Deshmukh, S., Xu, X., Mohammad, I. and Huang, H. (2011), "Antenna sensor skin for fatigue crack detection and monitoring", Smart Struct. Syst., Int. J., 8(1), 93-105. https://doi.org/10.12989/sss.2011.8.1.093
  6. He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 770-778. https://doi.org/10.1109/CVPR.2016.90
  7. He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), Mask R-CNN, pp. 2961-2969.
  8. Hoult, N.A., Fidler, P.R., Hill, P.G. and Middleton, C.R. (2010), "Wireless structural health monitoring of bridges: Present and future", Smart Struct. Syst., Int. J., 6(3), 277-290. https://doi.org/10.12989/sss.2010.6.3.277
  9. Jahanshahi, M.R. and Masri, S.F. (2013) "Effect of color space, color channels, and sub-image block size on the performance of wavelet-based texture analysis algorithms: An application to corrosion detection on steel structures", In: Computing in Civil Engineering - Proceedings of the 2013 ASCE International Workshop on Computing in Civil Engineering, pp. 685-692. https://doi.org/10.1061/9780784413029.086
  10. Jeong, Y., Kim, W., Lee, I. and Lee, J. (2018), "Bridge inspection practices and bridge management programs in China, Japan, Korea, and U.S.", J. Struct. Integr. Maint., 3(2), 126-135. https://doi.org/10.1080/24705314.2018.1461548
  11. Jiang, S., Wu, Y. and Zhang, J. (2023), "Bridge coating inspection based on two-stage automatic method and collision-tolerant unmanned aerial system", Automat. Constr., 146, 104685. https://doi.org/10.1016/j.autcon.2022.104685
  12. Jin Lim, H., Hwang, S., Kim, H. and Sohn, H. (2021), "Steel bridge corrosion inspection with combined vision and thermographic images", Struct. Health Monitor., 20(6), 3424-3435. https://doi.org/10.1177/1475921721989407
  13. Khayatazad, M., De Pue, L. and De Waele, W. (2020), "Detection of corrosion on steel structures using automated image processing", Develop. Built Environ., 3, p. 100022. https://doi.org/10.1016/j.dibe.2020.100022
  14. Koonce, B. (2021), "EfficientNet", In: Convolutional Neural Networks with Swift for Tensorflow, Berkeley, CA, USA, pp. 109-123. https://doi.org/10.1007/978-1-4842-6168-2_10
  15. Kreislova, K. and Geiplova, H. (2012), "Evaluation of corrosion protection of steel bridges", Procedia Eng., 40, 229-234. https://doi.org/10.1016/j.proeng.2012.07.085
  16. La, H.M., Dinh, T.H., Pham, N.H., Ha, Q.P. and Pham, A.Q. (2019), "Automated robotic monitoring and inspection of steel structures and bridges", Robotica, 37(5), 947-967. https://doi.org/10.1017/S0263574717000601
  17. Lee, J.H., Yoon, S., Kim, B., Gwon, G.H., Kim, I.H. and Jung, H.J. (2021), "A new image-quality evaluating and enhancing methodology for bridge inspection using an unmanned aerial vehicle", Smart Struct. Syst., Int. J., 27(2), 209-226. https://doi.org/10.12989/sss.2021.27.2.209
  18. Li, X. and Zhang, Y. (2008), "Feasibility study of wide-band low-profile ultrasonic sensor with flexible piezoelectric paint", Smart Struct. Syst., Int. J., 565-582. https://doi.org/10.12989/sss.2008.4.5.565
  19. Liao, K.W. and Lee, Y.T. (2016), "Detection of rust defects on steel bridge coatings via digital image recognition", Automat. Constr., 71(Part 2), 294-306. https://doi.org/10.1016/j.autcon.2016.08.008
  20. Margret, M., Menaka, M., Subramanian, V., Baskaran, R. and Venkatraman, B. (2018), "Non-destructive inspection of hidden corrosion through Compton backscattering technique", Radiat. Phys. Chem., 152, 158-164. https://doi.org/10.1016/j.radphyschem.2018.07.015
  21. Mazzinghi, A., Freni, A. and Capineri, L. (2019), "microwave non-destructive testing method for controlling polymeric coating of metal layers in industrial products', NDT E Int., 102, 207-217. https://doi.org/10.1016/j.ndteint.2018.12.003
  22. Ministry of Land (2019), Guidelines for the implementation of safety maintenance of infrastructure. https://www.mlit.go.jp/road/road_e/s3_maintenance.html
  23. Myung, H., Wang, Y., Kang, S.C. and Chen, X. (2014), "Survey on robotics and automation technologies for civil infrastructure". http://ir.canterbury.ac.nz/handle/10092/10781
  24. Ochiai, S., Iwamoto, S., Nakamura, T. and Okuda, H. (2007) "Crack spacing distribution in coating layer of galvannealed steel under applied tensile strain", ISIJ Int., 47(3), 458-465. https://doi.org/10.2355/isijinternational.47.458
  25. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. and Desmaison, A. (2019), PyTorch: An imperative style, high-performance deep learning library, Advances in Neural Information Processing Systems. http://papers.nips.cc/paper/9015-pytorch-an-imperative-stylehigh-
  26. Puspitasari, S.D. and Harahap, S. (2023), "Bridge inspection implementations and maintenance planning-A comparative analysis of a few distinctive countries", In: AIP Conference Proceedings, Vol. 2482, p. 50007. https://doi.org/10.1063/5.0110943
  27. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016), "You only look once: Unified, real-time object detection", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 779-788. https://doi.org/10.1109/CVPR.2016.91
  28. Shrestha, R. and Kim, W. (2018), "Evaluation of coating thickness by thermal wave imaging: A comparative study of pulsed and lock-in infrared thermography - Part II: Experimental investigation", Infrared Phys. Technol., 92, 24-29. https://doi.org/10.1016/j.infrared.2018.05.001
  29. Standard (1994), "Surface preparation and protective coating", MCr-501, Rev. 1(December), pp. 20-31. https://www.standard.no/pagefiles/1167/m-501.pdf
  30. Ulbrich, D. (2022), "Monitoring the boundary of an adhesive coating to a steel substrate with an ultrasonic Rayleigh wave", Open Eng., 12(1), 933-945. https://doi.org/10.1515/eng-2022-0383
  31. Zhu, Y. and Newsam, S. (2018), "DenseNet for dense flow", In: Proceedings - International Conference on Image Processing, ICIP, IEEE Computer Society, pp. 790-794. https://doi.org/10.1109/ICIP.2017.8296389