The Journal of The Korea Institute of Intelligent Transport Systems (한국ITS학회 논문지)

The Korea Institute of Intelligent Transport Systems (한국ITS학회)
권호 표지
DOI QR코드

DOI QR Code

CycleGAN Based Translation Method between Asphalt and Concrete Crack Images for Data Augmentation

데이터 증강을 위한 순환 생성적 적대 신경망 기반의 아스팔트와 콘크리트 균열 영상 간의 변환 기법

  • Shim, Seungbo (Korea Institute of Civil Engineering and Building Technology)
  • 심승보 (한국건설기술연구원 지반연구본부)
  • Received : 2022.09.17
  • Accepted : 2022.10.20
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

The safe use of a structure requires it to be maintained in an undamaged state. Thus, a typical factor that determines the safety of a structure is a crack in it. In addition, cracks are caused by various reasons, damage the structure in various ways, and exist in different shapes. Making matters worse, if these cracks are unattended, the risk of structural failure increases and proceeds to a catastrophe. Hence, recently, methods of checking structural damage using deep learning and computer vision technology have been introduced. These methods usually have the premise that there should be a large amount of training image data. However, the amount of training image data is always insufficient. Particularly, this insufficiency negatively affects the performance of deep learning crack detection algorithms. Hence, in this study, a method of augmenting crack image data based on the image translation technique was developed. In particular, this method obtained the crack image data for training a deep learning neural network model by transforming a specific case of a asphalt crack image into a concrete crack image or vice versa . Eventually, this method expected that a robust crack detection algorithm could be developed by increasing the diversity of its training data.

구조물을 안전하게 관리하기 위해서는 우선적으로 건전한 유지가 전제되어야 한다. 이 같은 구조물의 건전성을 결정하는 요인 중에서 가장 대표적인 예로는 균열을 들 수 있다. 여러 가지 원인에 의해 발생하는 균열은 다양한 종류와 형태로 구조물에 손상을 입힌다. 무엇보다 이러한 균열이 방치될 경우 위험도가 증가하여 안전사고로 이어질 수 있다. 이러한 문제점을 경감하기 위하여 최근 들어 딥러닝과 컴퓨터 비전 기술을 활용하여 손상을 점검하는 방법들이 소개되고 있다. 이 같은 방법들은 대체로 충분한 양의 학습 데이터가 필요한 것이 사실이다. 하지만, 학습을 위한 영상 데이터의 충분한 확보가 어렵다는 점은 딥러닝 균열 탐지 알고리즘의 성능에 부정적인 영향을 미친다. 따라서 본 논문에서는 이에 대한 문제의식을 바탕으로 영상 변환 기법을 활용하여 균열 영상 데이터를 증강하는 방법을 제시했다. 이는 아스팔트 균열 영상을 콘크리트 균열 영상으로 변환하거나 혹은 이와 반대로 콘크리트 균열 영상을 아스팔트 균열 영상으로 변환하여 딥러닝 신경망 모델을 학습하기 위한 영상 데이터를 확보하는 방법이다. 이를 통해 학습 데이터의 다양성을 향상시켜 강건한 균열 탐지 알고리즘 개발에 기여할 수 있기를 기대한다.

Keywords

Acknowledgement

이 성과는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구입니다(No. 2022R1F1A1074663). 지원에 감사합니다.

References

  1. Ali, R., Chuah, J. H., Talip, M. S. A., Mokhtar, N. and Shoaib, M. A.(2022), "Structural crack detection using deep convolutional neural networks", Automation in Construction, vol. 133, p.103989. https://doi.org/10.1016/j.autcon.2021.103989
  2. Amhaz, R., Chambon, S., Idier, J. and Baltazart, V.(2016), "Automatic crack detection on two-dimensional pavement images: An algorithm based on minimal path selection", IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 10, pp.2718-2729. https://doi.org/10.1109/TITS.2015.2477675
  3. Dellana, R. and Roy, K.(2016), "Data augmentation in CNN-based periocular authentication", In Proceedings of International Conference on Information Communication and Management(ICICM), Hatfield, UK, pp.141-145.
  4. Eisenbach, M., Stricker, R., Seichter, D., Amende, K., Debes, K., Sesselmann, M., Ebersbach, D., Stoeckert, U. and Gross, H. M.(2017), "How to get pavement distress detection ready for deep learning? A systematic approach", In Proceedings of International Joint Conference on Neural Networks(IJCNN), Anchorage, AK, USA, pp.2039-2047.
  5. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y.(2014), "Generative adversarial nets", In Proceedings of Advances in Neural Information Processing Systems(NIPS), Montreal, Canada, pp.2672-2680.
  6. Isola, P., Zhu, J. Y., Zhou, T. and Efros, A. A.(2017), "Image-to-image translation with conditional adversarial networks", In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA, pp.1125-1134.
  7. Karras, T., Laine, S. and Aila, T.(2019), "A style-based generator architecture for generative adversarial networks", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, CA, USA, pp.4401-4410.
  8. Krizhevsky, A., Sutskever, I. and Hinton, G. E.(2012), "Imagenet classification with deep convolutional neural networks", In Proceedings of Advances in Neural Information Processing Systems(NIPS), Montreal, Canada, vol. 25, pp.1097-1105.
  9. Long, L., Dohler, M. and Thons, S.(2022), "Determination of structural and damage detection system influencing parameters on the value of information", Structural Health Monitoring, vol. 21, no. 1, pp.19-36. https://doi.org/10.1177/1475921719900918
  10. Shi, Y., Cui, L., Qi, Z., Meng, F. and Chen, Z.(2016), "Automatic road crack detection using random structured forests", IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 12, pp.3434-3445. https://doi.org/10.1109/TITS.2016.2552248
  11. Shim, S., Kim, J., Cho, G. C. and Lee, S. W.(2022), "Stereo-vision-based 3D concrete crack detection using adversarial learning with balanced ensemble discriminator networks", Structural Health Monitoring, 14759217221097868.
  12. Shim, S., Kim, J., Lee, S. W. and Cho, G. C.(2021), "Road surface damage detection based on hierarchical architecture using lightweight auto-encoder network", Automation in Construction, vol. 130, p.103833. https://doi.org/10.1016/j.autcon.2021.103833
  13. Shorten, C. and Khoshgoftaar, T. M.(2019), "A survey on image data augmentation for deep learning", Journal of Big Data, vol. 6, no. 1, pp.1-48. https://doi.org/10.1186/s40537-018-0162-3
  14. Spencer Jr, B. F., Hoskere, V. and Narazaki, Y.(2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, vol, 5, no. 2, pp.199-222. https://doi.org/10.1016/j.eng.2018.11.030
  15. Taylor, L. and Nitschke, G.(2018), "Improving deep learning with generic data augmentation", In Proc. IEEE Symposium Series on Computational Intelligence(SSCI), Bangalore, India, pp.1542-1547.
  16. Thanoon, W. A., Jaafar, M. S., Kadir, M. R. A. and Noorzaei, J.(2005), "Repair and structural performance of initially cracked reinforced concrete slabs", Construction and Building Materials, vol. 19, no. 8, pp.595-603. https://doi.org/10.1016/j.conbuildmat.2005.01.011
  17. Wang, Z., Yang, J., Jiang, H. and Fan, X.(2020), "CNN training with twenty samples for crack detection via data augmentation", Sensors, vol. 20, no. 17, p.4849. https://doi.org/10.3390/s20174849
  18. Xu, B., Wang, N., Chen, T. and Li, M.(2015), Empirical evaluation of rectified activations in convolutional network, arXiv:1505.00853 [Online]. Available at https://arxiv.org/abs/1505.00853
  19. Zhang, L., Yang, F., Zhang, Y. D. and Zhu, Y. J.(2016), "Road crack detection using deep convolutional neural network", In Proceedings of IEEE International Conference on Image Processing(ICIP), Phoenix, AZ, USA, pp.3708-3712.
  20. Zhu, J. Y., Park, T., Isola, P. and Efros, A. A.(2017), "Unpaired image-to-image translation using cycle-consistent adversarial networks", In Proceedings of the IEEE International Conference on Computer Vision(ICCV), Venice, Italy, pp.2223-2232.
  21. Zou, Q., Cao, Y., Li, Q., Mao, Q. and Wang, S.(2012), "CrackTree: Automatic crack detection from pavement images", Pattern Recognition Letters, vol. 33, no. 3, pp.227-238. https://doi.org/10.1016/j.patrec.2011.11.004