Smart Structures and Systems

  • 제34권2호
  • /
  • Pages.129-143
  • /
  • 2024
  • /
  • 1738-1584(pISSN)
  • /
  • 1738-1991(eISSN)
테크노프레스 (Techno-Press)
권호 표지
DOI QR코드

DOI QR Code

Generation of wind turbine blade surface defect dataset based on StyleGAN3 and PBGMs

  • W.R. Li (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • W.H. Zhao (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • T.T. Wang (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • Y.F. Du (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology)
  • 투고 : 2024.04.16
  • 심사 : 2024.09.30
  • 발행 : 2024.08.25
⟨ 이전 논문 다음 논문 ⟩

초록

In recent years, with the vigorous development of visual algorithms, a large amount of research has been conducted on blade surface defect detection methods represented by deep learning. Detection methods based on deep learning models must rely on a large and rich dataset. However, the geographical location and working environment of wind turbines makes it difficult to effectively capture images of blade surface defects, which inevitably hinders visual detection. In response to the challenge of collecting a dataset for surface defects that are difficult to obtain, a multi-class blade surface defect generation method based on the StyleGAN3 (Style Generative Adversarial Networks 3) deep learning model and PBGMs (Physics-Based Graphics Models) method has been proposed. Firstly, a small number of real blade surface defect datasets are trained using the adversarial neural network of the StyleGAN3 deep learning model to generate a large number of high-resolution blade surface defect images. Secondly, the generated images are processed through Matting and Resize operations to create defect foreground images. The blade background images produced using PBGM technology are randomly fused, resulting in a diverse and high-resolution blade surface defect dataset with multiple types of backgrounds. Finally, experimental validation has proven that the adoption of this method can generate images with defect characteristics and high resolution, achieving a proportion of over 98.5%. Additionally, utilizing the EISeg annotation method significantly reduces the annotation time to just 1/7 of the time required for traditional methods. These generated images and annotated data of blade surface defects provide robust support for the detection of blade surface defects.

키워드

과제정보

The research described in this paper was financially supported by the National Science Foundation of China (Grant Nos. 52068049 and 51908266), the Science Fund for Distinguished Young Scholars of Gansu Province (No. 21JR7RA267), and Hongliu Outstanding Young Talents Program of Lanzhou University of Technology.

참고문헌

  1. Amenabar, I., Mendikute, A., Lopez-Arraiza, A., Lizaranzu, M. and Aurrekoetxea, J. (2011), "Comparison and analysis of nondestructive testing techniques suitable for delamination inspection in wind turbine blades", Compos. B. Eng., 42(5), 1298-1305. https://doi.org/10.1016/j.compositesb.2011.01.025
  2. Arjovsky, M. and Bottou, L. (2017), "Towards principled methods for training generative adversarial networks", Proc. ICLR, 1-17. https://doi.org/10.48550/arXiv.1701.04862
  3. Beganovic, N. and Soffker, D. (2016), "Structural health management utilization for lifetime prognosis and advanced control strategy deployment of wind turbines: An overview and outlook concerning actual methods, tools, and obtained results", Renew. Sustain. Energy Rev., 64, 68-83. https://doi.org/10.1016/j.rser.2016.05.083
  4. Chu, M., Xie, Y., Mayer, J., Leal-Taixe, L. and Thuerey, N. (2020), "Learning temporal coherence via self-supervision for GAN-based video generation", ACM Trans. Graph., 39(4), 75:1-75:13. https://doi.org/10.1145/3386569.3392457
  5. Du, Y., Zhou, S., Jing, X., Peng, Y., Wu, H. and Kwok, N. (2020), "Damage detection techniques for wind turbine blades: A review", Mech. Syst. Signal Process., 141, 106445. https://doi.org/10.1016/j.ymssp.2019.106445
  6. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., WardeFarley, D., Ozair, S., Courville, A. and Bengio, Y. (2014), "Generative adversarial nets", Proc. NIPS, 27, 2672-2680. https://doi.org/10.12989/sss.2023.31.5.469
  7. Guo, J., Liu, C., Cao, J. and Jiang, D. (2021), "Damage identification of wind turbine blades with deep convolutional neural networks", Renew. Energy, 174, 122-133. https://doi.org/10.1016/j.renene.2021.04.040
  8. Hernandez-Estrada, E., Lastres-Danguillecourt, O., Robles-Ocampo, J.B., Lopez-Lopez, A., Sevilla-Camacho, P.Y., Perez-Sarinana, B.Y. and Dorrego-Portela, J.R. (2021), "Considerations for the structural analysis and design of wind turbine towers: A review", Renew. Sust. Energ. Rev., 137, 110447. https://doi.org/10.1016/j.rser.2020.110447
  9. Hoskere, V., Narazaki, Y. and Spencer Jr, B.F. (2022), "Physics-based graphics models in 3D synthetic environments as autonomous vision-based inspection testbeds", Sensors, 22(2), 532. https://doi.org/10.3390/s22020532
  10. Kaewniam, P., Cao, M., Alkayem, N.F., Li, D. and Manoach, E. (2022), "Recent advances in damage detection of wind turbine blades: A state-of-the-art review", Renew. Sust. Energ. Rev., 167, 112723. https://doi.org/10.1016/j.rser.2022.112723
  11. Karras, T., Laine, S. and Aila, T. (2019), "A style-based generator architecture for generative adversarial networks", In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, June.
  12. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J. and Aila, T. (2020), "Analyzing and improving the image quality of stylegan", In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Seattle, WA, USA, June.
  13. Karras, T., Aittala, M., Laine, S., Harkonen E., Hellsten, J., Lehtinen, J. and Aila. T. (2021), "Alias-free generative adversarial networks", Adv. Neural Inf. Process. Syst., 34, 852-863.
  14. Liu, Y., Chu, L., Chen, G., Wu, Z., Chen, Z., Lai, B. and Hao, Y. (2021), "Paddleseg: A high-efficient development toolkit for image segmentation", arXiv preprint arXiv:2101.06175 [cs.LG]. https://doi.org/10.48550/arXiv.2101.06175
  15. Moreno, S., Pena, M., Toledo, A., Trevino, R. and Ponce, H. (2018), "A new vision-based method using deep learning for damage inspection in wind turbine blades", In: The 15th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), Mexico City, Mexico, September.
  16. Narazaki, Y., Hoskere, V., Eick, B.A., Smith, M.D. and Spencer, B.F. (2019), "Vision-based dense displacement and strain estimation of miter gates with the performance evaluation using physics-based graphics models", Smart Struct. Syst., Int. J., 24(6), 709-721. https://doi.org/10.12989/sss.2019.24.6.709
  17. Oliveira, G., Magalhaes, F., Cunha, A. and Caetano, E. (2018), "Continuous dynamic monitoring of an onshore wind turbine", Eng. Struct., 164, 22-39. https://doi.org/10.1016/j.engstruct.2018.02.030
  18. Ozbek, M., Meng, F. and Rixen, D.J. (2013), "Challenges in testing and monitoring the in-operation vibration characteristics of wind turbines", Mech. Syst. Signal Process, 41(1-2), 649-666. https://doi.org/10.1016/j.ymssp.2013.07.023
  19. Park, T., Zhu, J.Y., Wang, O., Lu, J., Shechtman, E., Efros, A.A. and Zhang, R. (2017), "Swapping autoencoder for deep image manipulation", Adv. Neural. Inf. Process. Syst., 33, 7198-7211.
  20. Park, T., Liu, M.Y., Wang, T.C. and Zhu, J.Y. (2019), "Semantic image synthesis with spatially-adaptive normalization", In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, June.
  21. Radford, A., Metz, L. and Chintala, S. (2016), "Unsupervised representation learning with deep convolutional generative adversarial networks", arXiv:1511.06434 [cs.LG]. https://doi.org/10.48550/arXiv.1511.06434
  22. Ruiz, M., Mujica, L.E., Alferez, S., Acho, L., Tutiven, L., Vidal, Y., Rodellar, J. and Pozo, F. (2018), "Wind turbine fault detection and classification by means of image texture analysis", Mech. Syst. Signal Process, 107, 149-167. https://doi.org/10.1016/j.ymssp.2017.12.035
  23. Sarkar, D. and Gunturi, S.K. (2021), "Wind turbine blade structural state evaluation by hybrid object detector relying on deep learning models", J. Amb. Intel. Hum. Comp., 12, 8535-8548. https://doi.org/10.1007/s12652-020-02587-7
  24. Sony, S., Laventure, S. and Sadhu, A. (2019), "A literature review of next-generation smart sensing technology in structural health monitoring", Struct. Control Health Monit., 26(3), e2321. https://doi.org/10.1002/stc.2321
  25. Sun, S., Wang, T., Yang, H. and Chu, F. (2022), "Damage identification of wind turbine blades using an adaptive method for compressive beamforming based on the generalized minimax-concave penalty function", Renew. Energy, 181, 59-70. https://doi.org/10.1016/j.renene.2021.09.024
  26. Wang, L. and Zhang, Z. (2017), "Automatic detection of wind turbine blade surface cracks based on UAV-taken images", IEEE Trans. Ind. Electron, 64(9), 7293-7303. https://doi.org/10.1109/TIE.2017.2682037
  27. Wang, L., Zhang, Z. and Luo, X. (2019), "A two-stage data-driven approach for image-based wind turbine blade crack inspections", IEEE ASME Trans. Mechatron, 24(3), 1271-1281. https://doi.org/10.1109/TMECH.2019.2908233
  28. Xu, D., Wen, C. and Liu, J. (2019), "Wind turbine blade surface inspection based on deep learning and UAV-taken images", J. Renew Sustain. Energy, 11(5), 053305. https://doi.org/10.1063/1.5113532
  29. Yang, B. and Sun, D. (2013), "Testing, inspecting and monitoring technologies for wind turbine blades: A survey", Renew. Sustain. Energy Rev., 22, 515-526. https://doi.org/10.1016/j.rser.2012.12.056
  30. Yang, X., Zhang, Y., Lv, W. and Wang, D. (2021), "Image recognition of wind turbine blade damage based on a deep learning model with transfer learning and an ensemble learning classifier", Renew. Energy, 163, 386-397. https://doi.org/10.1016/j.renene.2020.08.125
  31. Yu, Y., Cao, H., Yan, X., Wang, T. and Ge, S.S. (2020), "Defect identification of wind turbine blades based on defect semantic features with transfer feature extractor", Neurocomputing, 367, 1-9. https://doi.org/10.1016/j.neucom.2019.09.071
  32. Zhao, W.H., Li, W.R., Yang, M.H., Hong, Na. and Du, Y.F. (2023), "Dynamic characteristics monitoring of wind turbine blades based on improved YOLOv5 deep learning model", Smart Struct. Syst., Int. J., 31(5), 469-483. https://doi.org/10.12989/sss.2023.31.5.469
  33. Zhou, H.F., Zheng, J.F., Xie, Z.L., Lu, L.J., Ni, Y.Q. and Ko, J.M. (2017), "Temperature effects on vision measurement system in long-term continuous monitoring of displacement", Renew. Energy, 114, 968-983. https://doi.org/10.1016/j.renene.2017.07.104
  34. Zhu, X., Hang, X., Gao, X., Yang, X., Xu, Z., Wang, Y. and Liu, H. (2022), "Research on crack detection method of wind turbine blade based on a deep learning method", Appl. Energy, 328, 120241. https://doi.org/10.1016/j.apenergy.2022.120241