Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Development and testing of a composite system for bridge health monitoring utilising computer vision and deep learning

  • Lydon, Darragh (School of Natural and Built Environment, Queen's University) ;
  • Taylor, S.E. (School of Natural and Built Environment, Queen's University) ;
  • Lydon, Myra (School of Natural and Built Environment, Queen's University) ;
  • Martinez del Rincon, Jesus (School of Electronics, Electrical Engineering and Computer Sciences, Queen's University) ;
  • Hester, David (School of Natural and Built Environment, Queen's University)
  • Received : 2019.07.12
  • Accepted : 2019.08.30
  • Published : 2019.12.25
⟨ Previous Next ⟩

Abstract

Globally road transport networks are subjected to continuous levels of stress from increasing loading and environmental effects. As the most popular mean of transport in the UK the condition of this civil infrastructure is a key indicator of economic growth and productivity. Structural Health Monitoring (SHM) systems can provide a valuable insight to the true condition of our aging infrastructure. In particular, monitoring of the displacement of a bridge structure under live loading can provide an accurate descriptor of bridge condition. In the past B-WIM systems have been used to collect traffic data and hence provide an indicator of bridge condition, however the use of such systems can be restricted by bridge type, assess issues and cost limitations. This research provides a non-contact low cost AI based solution for vehicle classification and associated bridge displacement using computer vision methods. Convolutional neural networks (CNNs) have been adapted to develop the QUBYOLO vehicle classification method from recorded traffic images. This vehicle classification was then accurately related to the corresponding bridge response obtained under live loading using non-contact methods. The successful identification of multiple vehicle types during field testing has shown that QUBYOLO is suitable for the fine-grained vehicle classification required to identify applied load to a bridge structure. The process of displacement analysis and vehicle classification for the purposes of load identification which was used in this research adds to the body of knowledge on the monitoring of existing bridge structures, particularly long span bridges, and establishes the significant potential of computer vision and Deep Learning to provide dependable results on the real response of our infrastructure to existing and potential increased loading.

Keywords

References

  1. ACSE. (2017), "Report card for America's infrastructure", New York, NY, USA. URL:https://www.infrastructurereportcard.org/cat-item/bridges/
  2. Arinaldi, A., Jaka, A.P. and Arlan, A.G. (2018), "Detection and classification of vehicles for traffic video analytics", Pro. C. Sci., 144(1), 259-268. DOI: 10.1016/J.PROCS.2018.10.527.
  3. Cavaioni, M. (2018), "Deep learning series: Convolutional neural networks - machine learning bites - medium", Medium/Machine Learning. 2018. URL:https://medium.com/machine-learning-bites/deeplearningseries-convolutional-neural-networks-a9c2f2ee1524
  4. Dong, C.Z., Celik, O. and Catbas, F.N. (2018), "Marker-free monitoring of the grandstand structures and modal identification using computer vision methods", Struct. Health Monit., DOI: 10.1177/1475921718806895.
  5. Feng, M.Q., Feng, D.M, Fukuda, Y. and Mizuto M. (2015), "Nontarget vision sensor for remote measurement of bridge dynamic response", J. Bridge Eng., 20(12). DOI:10.1061/(ASCE)BE.1943-5592.0000747.
  6. Michael, F., Ahmed, E., He, X.F. and Conte, J.P. (2010), "Sensor network for structural health monitoring of a highway bridge", J. Comput. Civil Eng., 24(1), 11-24. DOI:10.1061/(ASCE)CP.1943-5487.0000005.
  7. Fukuda, Y., Feng, M.Q. and Masanobu S. (2010), "Cost-effective vision-based system for monitoring dynamic response of civil engineering structures", Struct. Control Hwalth. Monit., 17(8), 918-936. DOI: 10.1002/stc.360.
  8. Graybeal, B.A., Phares, B.M., Rolander, D.D., Mark, M. and Glenn, W. (2002), "Visual inspection of highway bridges", J. Nondestruct. Eval., 21(3), 67-83. DOI:10.1023/A:1022508121821.
  9. Hubel, D.H. and Wiesel T.N. (1962), "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex", J. Physiol., 160(1), 106-154. DOI:10.1113/jphysiol.1962.sp006837.
  10. Lydon,D., Lydon, M., Del Rincon, J.M., Taylor, S.E., Robinson, D., O'Brien, E. and Catbas, F.N. (2018), "Development and field testing of a time-synchronized system for multi-point displacement calculation using low-cost wireless vision-based sensors", IEEE Sens. J., 18(23), 9744-9754. DOI:10.1109/JSEN.2018.2853646.
  11. Marcin, M., Krzysztof, M., Jakub, P., Malgorzata, K., Pawel, S., Artur, P., Tomasz, L. and Andrzej, Z. (2016), "Non-destructive testing of industrial structures with the use of multi-camera digital image correlation method", Eng. Fail. Anal., 69(11), 122-134. DOI: 10.1016/j.engfailanal.2016.02.002.
  12. Krzysztof, M., Marcin, M., Tomasz, K. and Malgorzata, K. (2017), "Multi-camera digital image correlation method with distributed fields of view", Opt. Laser. Eng., 98(11), 198-204. DOI: 10.1016/J.OPTLASENG.2017.05.003.
  13. Li, S.H., Lin, J.Z., Li, G.Q., Bai, T., Wang, H.Q. and Pang, Y. (2018), "Vehicle type detection based on deep learning in traffic scene", Pro. C. Sci., 131(2018), 564-572. DOI:10.1016/J.PROCS.2018.04.281
  14. OECD (2016), "Transport infrastructure investment and maintenance spending", Paris, France. URL:https://stats.oecd.org/
  15. Office of National Statistics (2018), "Developing new statistics of infrastructure: August 2018", London, UK. URL:https://www.ons.gov.uk/economy/economicoutputandproductivity/productivitymeasures/articles/developingnewmeasuresofinfrastructureinvestment/august2018
  16. Ojio, T., Carey, C.H., OBrien, E.J., Doherty, C. and Taylor, S.E. (2016), "Contactless bridge weigh-in-motion", J. Bridge Eng., 21(7), 4016032. DOI: 10.1061/(ASCE)BE.1943-5592.0000776.
  17. Park, J.W., Lee, J.J., Jung, H.J. and Myung, H. (2010), "Visionbased displacement measurement method for high-rise building structures using partitioning approach", NDT & E Int., 43(7), 642-647. DOI: 10.1016/j.ndteint.2010.06.009.
  18. Piccardi, M. (2004), "Background subtraction techniques: A review", IEEE Inter. Confer. Sys., 4, 3099-3104. DOI:10.1109/ICSMC.2004.1400815.
  19. RAC Foundation (2019), "Bridge maintenance backlog grows", London, UK. URL: https://www.racfoundation.org/mediacentre/bridge-maintenance-backlog-grows
  20. Redmon, J. and Farhadi, A. (2018), "YOLOv3: An incremental improvement." arXiv:1804.02767, DOI:10.1109/CVPR.2017.690.
  21. Ren, S.Q., He, K.M., Girshick, R. and Sun, J. (2015), "Faster RCNN:Towards real-time object detection with region proposal networks", IEEE T. Pattern Anal., 39(6), 1137-1149. DOI:10.1109/TPAMI.2016.2577031.
  22. Ward, R. and Jakob D.H. (2007), "Public capital and economic growth: A critical survey", Per. Der Wirts., 8(1), 6-52. DOI:10.1111/1468-2516.00242.
  23. See, J.E. (2012), "Visual inspection: A review of the literature." Off. Sci. Tech. Inform., 1-77. URL:https://pdfs.semanticscholar.org/5913/d3cbf8c389a8fdfe3fd5f2e880a0da03ba5a.pdf
  24. Simonyan, K. and Andrew, Z. (2015), "Very deep convolutional networks for large-scale image recognition", CoRR abs/1409.1556., DOI: arXiv:1409.1556
  25. Stephen, G.A., Brownjohn, J.M.W. and Taylor, C.A. (1993). "Measurements of static and dynamic displacement from visual monitoring of the Humber Bridge", Eng. Struct., 15(3):197-208. DOI: 10.1016/0141-0296(93)90054-8.
  26. Szegedy, C., Liu, W., Jia, Y.P., Pierre, S., Scott, R., Dragomir, A., Dumitru, E., Vincent, V. and Andrew, R. (2015), "Going deeper with convolutions", IEEE CVPR., DOI:10.1109/CVPR.2015.7298594.
  27. World Economic Forum (2018), "The global competitiveness report 2018", Cologny, Switzerland. URL:http://reports.weforum.org/global-competitiveness-report-2018/competitiveness-rankings/#series=ROADINF
  28. Worth, D. (2010), "Introduction to modern information retrieval", Australian Academic and Research Libraries., 41(4), 305-306. DOI: 10.1080/00048623.2010.10721488.
  29. Ye, X.W., Ni, Y.Q., Wai T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013), "A vision-based system for dynamic displacement measurement of long-span bridges: Algorithm and verification", Smart Struct. Syst., 12(3-4), 363-379. https://doi.org/10.12989/sss.2013.12.3-4.363.
  30. Ye, X.W., Yi, T.H., Wen, C. and Su, Y.H. (2015), "Reliabilitybased assessment of steel bridge deck using a mesh-insensitive structural stress method", Smart. Struct. Syst., 16(2), 367-382. https://doi.org/10.12989/sss.2015.16.2.367.
  31. Ye, X.W., Dong, C.Z. and Liu, T. (2016), "Image-based structural dynamic displacement measurement using different multiobject tracking algorithms", Smart Struct Syst., 17(6), 935-956. https://doi.org/10.12989/sss.2016.17.6.935.
  32. Zaurin, R. and Catbas, F.N. (2011), "Structural health monitoring using video stream, influence lines, and statistical analysis", Struct. Health Monit., 10(3), 309-332. DOI:10.1177/1475921710373290.
  33. Zaurin, R., Khuc, T. and Catbas, F.N. (2015), "Hybrid sensorcamera monitoring for damage detection: Case study of a real bridge", J. Bridge Eng., 21(6), 1-27. DOI:10.1061/(ASCE)BE.1943.
  34. Zaurin, R. and Catbas, F.N. (2011). "Structural health monitoring using video stream, influence lines, and statistical analysis", Struct. Health Monit., 10(3), 309-332. DOI:10.1177/1475921710373290.
  35. Zhuo, L., Jiang, L.Y., Zhu, Z.Q., Li, J.F., Zhang, J., and Long, H.X. (2017), "Vehicle classification for large-scale traffic surveillance videos using convolutional neural networks", Mach. Vision. Appl., 28(7), 793-802. DOI: 10.1007/s00138-017-0846-2.