Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

A novel computer vision-based vibration measurement and coarse-to-fine damage assessment method for truss bridges

  • Wen-Qiang Liu (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • En-Ze Rui (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Lei Yuan (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Si-Yi Chen (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • You-Liang Zheng (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University) ;
  • Yi-Qing Ni (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
  • Received : 2022.08.30
  • Accepted : 2023.02.02
  • Published : 2023.04.25
⟨ Previous Next ⟩

Abstract

To assess structural condition in a non-destructive manner, computer vision-based structural health monitoring (SHM) has become a focus. Compared to traditional contact-type sensors, the advantages of computer vision-based measurement systems include lower installation costs and broader measurement areas. In this study, we propose a novel computer vision-based vibration measurement and coarse-to-fine damage assessment method for truss bridges. First, a deep learning model FairMOT is introduced to track the regions of interest (ROIs) that include joints to enhance the automation performance compared with traditional target tracking algorithms. To calculate the displacement of the tracked ROIs accurately, a normalized cross-correlation method is adopted to fine-tune the offset, while the Harris corner matching is utilized to correct the vibration displacement errors caused by the non-parallel between the truss plane and the image plane. Then, based on the advantages of the stochastic damage locating vector (SDLV) and Bayesian inference-based stochastic model updating (BI-SMU), they are combined to achieve the coarse-to-fine localization of the truss bridge's damaged elements. Finally, the severity quantification of the damaged components is performed by the BI-SMU. The experiment results show that the proposed method can accurately recognize the vibration displacement and evaluate the structural damage.

Keywords

Acknowledgement

The work was supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region (SAR), China (Grant No. R5020-18) and a grant from The Hong Kong Polytechnic University (Grant No. 1-YW5H). The authors also appreciate the funding support by the Innovation and Technology Commission of Hong Kong SAR Government to the Hong Kong Branch of National Engineering Research Center on Rail Transit Electrification and Automation (Grant No. K-BBY1), and The Hong Kong Polytechnic University's Postdoc Matching Fund Scheme (Grant No. 1-W20D).

References

  1. An, Y., Ou, J., Li, J. and Spencer, B. (2014), "Stochastic DLV method for steel truss structures: simulation and experiment", Smart Struct. Syst., Int. J., 14(2), 105-128. https://doi.org/10.12989/sss.2014.14.2.105
  2. Bakhtiari-Nejad, F., Rahai, A. and Esfandiari, A. (2005), "A structural damage detection method using static noisy data", Eng. Struct., 27(12), 1784-1793. https://doi.org/10.1016/j.engstruct.2005.04.019
  3. Bernagozzi, G., Mukhopadhyay, S., Betti, R., Landi, L. and Diotallevi, P.P. (2018), "Output-only damage detection in buildings using proportional modal flexibility-based deflections in unknown mass scenarios", Eng. Struct., 167, 549-566. https://doi.org/10.1016/j.engstruct.2018.04.036
  4. Bernagozzi, G., Achilli, A., Betti, R., Diotallevi, P.P., Landi, L., Quqa, S. and Tronci, E.M. (2021), "On the use of multivariate autoregressive models for vibration-based damage detection and localization", Smart Struct. Syst., Int. J., 27(2), 335-350. https://doi.org/10.12989/sss.2021.27.2.335
  5. Bernal, D. (2002), "Load vectors for damage localization", J. Eng. Mech., 128(1), 7-14. https://doi.org/10.1061/(ASCE)0733-9399(2002)128:1(7)
  6. Bernal, D. (2006), "Flexibility-based damage localization from stochastic realization results", J. Eng. Mech., 132(6), 651-658. https://doi.org/10.1061/(ASCE)0733-9399(2006)132:6(651)
  7. Cha, Y.-J., Chen, J.G. and Buyukozturk, O. (2017), "Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters", Eng. Struct., 132, 300-313. https://doi.org/10.1016/j.engstruct.2016.11.038
  8. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H. and Wei, Y. (2017), "Deformable convolutional networks", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October.
  9. Dong, C.-Z. and Catbas, F.N. (2021), "A review of computer vision-based structural health monitoring at local and global levels", Struct. Health Monit., 20(2), 692-743. https://doi.org/10.1177/1475921720935585
  10. Eltouny, K.A. and Liang, X. (2021), "Bayesian-optimized unsupervised learning approach for structural damage detection", Comput.-Aided Civ. Infrastr. Eng., 36(10), 1249-1269. https://doi.org/10.1111/mice.12680
  11. Erdogan, Y.S. and Ada, M. (2020), "A computer-vision based vibration transducer scheme for structural health monitoring applications", Smart Mater. Struct., 29(8), 085007. https://doi.org/10.1088/1361-665X/ab9062
  12. Fang, K., Xiang, Y., Li, X. and Savarese, S. (2018), "Recurrent autoregressive networks for online multi-object tracking", Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, USA, March.
  13. Feng, D. and Feng, M.Q. (2016), "Output-only damage detection using vehicle-induced displacement response and mode shape curvature index", Struct. Control Health Monit., 23(8), 1088-1107. https://doi.org/10.1002/stc.1829
  14. Feng, D. and Feng, M.Q. (2017), "Experimental validation of cost-effective vision-based structural health monitoring", Mech. Syst. Sig. Process., 88, 199-211. https://doi.org/10.1016/j.ymssp.2016.11.021
  15. Figueiredo, E., Radu, L., Worden, K. and Farrar, C.R. (2014), "A Bayesian approach based on a Markov-chain Monte Carlo method for damage detection under unknown sources of variability", Eng. Struct., 80, 1-10. https://doi.org/10.1016/j.engstruct.2014.08.042
  16. Frans, R., Arfiadi, Y. and Parung, H. (2017), "Comparative study of mode shapes curvature and damage locating vector methods for damage detection of structures", Procedia Eng., 171, 1263-1271. https://doi.org/10.1016/j.proeng.2017.01.420
  17. Gao, Y., Spencer Jr, B.F. and Bernal, D. (2007), "Experimental verification of the flexibility-based damage locating vector method", J. Eng. Mech., 133(10), 1043-1049. https://doi.org/10.1061/(ASCE)0733-9399(2007)133:10(1043)
  18. Gomez, F., Narazaki, Y., Hoskere, V., Spencer Jr, B.F. and Smith, M.D. (2022), "Bayesian inference of dense structural response using vision-based measurements", Eng. Struct., 256, 113970. https://doi.org/10.1016/j.engstruct.2022.113970
  19. He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, June.
  20. He, Y., Zhang, L., Chen, Z. and Li, C.Y. (2022), "A framework of structural damage detection for civil structures using a combined multi-scale convolutional neural network and echo state network", Eng. Comput., 1-19. https://doi.org/10.1007/s00366-021-01584-4
  21. Khuc, T. and Catbas, F.N. (2017), "Completely contactless structural health monitoring of real-life structures using cameras and computer vision", Struct. Control Health Monit., 24(1), e1852. https://doi.org/10.1002/stc.1852
  22. Kim, C.-W., Morita, T., Oshima, Y. and Sugiura, K. (2015), "A Bayesian approach for vibration-based long-term bridge monitoring to consider environmental and operational changes", Smart Struct. Syst., Int. J., 15(2), 395-408. https://doi.org/10.12989/sss.2015.15.2.395
  23. Lei, Y., Liu, L., Mi, J. and Zhang, Y. (2021), "Damage detection of bridge structures under unknown seismic excitations using support vector machine based on transmissibility function and wavelet packet energy", Smart Struct. Syst., Int. J., 27(2), 257-266. https://doi.org/10.12989/sss.2021.27.2.257
  24. Liang, X. (2019), "Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization", Comput.-Aided Civ. Infrastr. Eng., 34(5), 415-430. https://doi.org/10.1111/mice.12425
  25. Mahmoudi, N., Ahadi, S.M. and Rahmati, M. (2019), "Multi-target tracking using CNN-based features: CNNMTT", Multimedia Tools Applicat., 78(6), 7077-7096. https://doi.org/10.1007/s11042-018-6467-6
  26. Mousavi, A.A., Zhang, C., Masri, S.F. and Gholipour, G. (2022), "Structural damage detection method based on the complete ensemble empirical mode decomposition with adaptive noise: a model steel truss bridge case study", Struct. Health Monit., 21(3), 887-912. https://doi.org/10.1177/14759217211013535
  27. Nagarajaiah, S. and Basu, B. (2009), "Output only modal identification and structural damage detection using time frequency & wavelet techniques", Earthq. Eng. Eng. Vib., 8(4), 583-605. https://doi.org/10.1007/s11803-009-9120-6
  28. Narazaki, Y., Gomez, F., Hoskere, V., Smith, M.D. and Spencer Jr, B.F. (2021), "Efficient development of vision-based dense three-dimensional displacement measurement algorithms using physics-based graphics models", Struct. Health Monit., 20(4), 1841-1863. https://doi.org/10.1177/1475921720939522
  29. Ngeljaratan, L. and Moustafa, M.A. (2020), "Structural health monitoring and seismic response assessment of bridge structures using target-tracking digital image correlation", Eng. Struct., 213, 110551. https://doi.org/10.1016/j.engstruct.2020.110551
  30. Ni, Y.Q. and Zhang, Q.H. (2021), "A Bayesian machine learning approach for online detection of railway wheel defects using track-side monitoring", Struct. Health Monit., 20(4), 1536-1550. https://doi.org/10.1177/1475921720921772
  31. Ni, Y.Q., Wang, J. and Chan, T.H.T. (2015), "Structural damage alarming and localization of cable-supported bridges using multi-novelty indices: a feasibility study", Struct. Eng. Mech., Int. J., 54(2), 337-362. https://doi.org/10.12989/sem.2015.54.2.337
  32. Ni, Y.Q., Wang, Y.W., Liao, W.Y. and Chen, W.H. (2019), "A vision-based system for long-distance remote monitoring of dynamic displacement: experimental verification on a supertall structure", Smart Struct. Syst., Int. J., 24(6), 769-781. https://doi.org/10.12989/sss.2019.24.6.769
  33. Ni, Y.Q., Wang, Y.W. and Zhang, C. (2020), "A Bayesian approach for condition assessment and damage alarm of bridge expansion joints using long-term structural health monitoring data", Eng. Struct., 212, 110520. https://doi.org/10.1016/j.engstruct.2020.110520
  34. Shi, J. (1994), "Good features to track", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, June.
  35. Spencer Jr, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  36. Sun, L., Shang, Z., Xia, Y., Bhowmick, S. and Nagarajaiah, S. (2020), "Review of bridge structural health monitoring aided by big data and artificial intelligence: From condition assessment to damage detection", J. Struct. Eng., 146(5), 04020073. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002535
  37. Uzun, M., Sun, H., Smit, D. and Buyukozturk, O. (2019), "Structural damage detection using Bayesian inference and seismic interferometry", Struct. Control Health Monit., 26(11), e2445. https://doi.org/10.1002/stc.2445
  38. Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B.B.G., Geiger, A. and Leibe, B. (2019), "Mots: Multi-object tracking and segmentation", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, June.
  39. Wan, H.P. and Ren, W.X. (2016), "Stochastic model updating utilizing Bayesian approach and Gaussian process model", Mech. Syst. Sig. Process., 70, 245-268. https://doi.org/10.1016/j.ymssp.2015.08.011
  40. Wang, J., Liu, X.Z. and Ni, Y.Q. (2018), "A Bayesian probabilistic approach for acoustic emission-based rail condition assessment", Comput.-Aided Civ. Infrast. Eng., 33(1), 21-34. https://doi.org/10.1111/mice.12316
  41. Wang, Z., Zheng, L., Liu, Y., Li, Y. and Wang, S. (2020), "Towards real-time multi-object tracking", Proceedings of the European Conference on Computer Vision, Glasgow, UK, August.
  42. Wang, X., Hou, R., Xia, Y. and Zhou, X. (2021a), "Structural damage detection based on variational Bayesian inference and delayed rejection adaptive Metropolis algorithm", Struct. Health Monit., 20(4), 1518-1535. https://doi.org/10.1177/1475921720921256
  43. Wang, Y.W., Ni, Y.Q., Zhang, Q.H. and Zhang, C. (2021b), "Bayesian approaches for evaluating wind-resistant performance of long-span bridges using structural health monitoring data", Struct. Control Health Monit., 28(4), e2699. https://doi.org/10.1002/stc.2699
  44. Wang, Q.A., Dai, Y., Ma, Z.G., Ni, Y.Q., Tang, J.Q., Xu, X.Q. and Wu, Z.Y. (2022a), "Towards probabilistic data-driven damage detection in SHM using sparse Bayesian learning scheme", Struct. Control Health Monit., 29(11), e3070. https://doi.org/10.1002/stc.3070
  45. Wang, Y.W., Ni, Y.Q. and Wang, S.M. (2022b), "Structural health monitoring of railway bridges using innovative sensing technologies and machine learning algorithms: a concise review", Intell. Transport. Infrastr., 1, liac009. https://doi.org/10.1093/iti/liac009
  46. Wang, Y.W., Zhang, C., Ni, Y.Q. and Xu, X.Y. (2022c), "Bayesian probabilistic assessment of occupant comfort of high-rise structures based on structural health monitoring data", Mech. Syst. Sig. Process., 163, 108147. https://doi.org/10.1016/j.ymssp.2021.108147
  47. Xu, Y. (2020), "Photogrammetry-based structural damage detection by tracking a visible laser line", Struct. Health Monit., 19(1), 322-336. https://doi.org/10.1177%2F1475921719840354 https://doi.org/10.1177%2F1475921719840354
  48. Xu, Y., Brownjohn, J. and Kong, D. (2018), "A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge", Struct. Control Health Monit., 25(5), e2155. https://doi.org/10.1002/stc.2155
  49. Ye, X.W., Ni, Y.Q., Wai, T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013), "A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification", Smart Struct. Syst., Int. J., 12(3-4), 363-379. https://doi.org/10.12989/sss.2013.12.3_4.363
  50. Ye, X.W., Dong, C.Z. and Liu, T. (2016), "A review of machine vision-based structural health monitoring: methodologies and applications", J. Sens., 2016. https://doi.org/10.1155/2016/7103039
  51. Zhang, Y., Wang, C., Wang, X., Zeng, W. and Liu, W. (2021), "Fairmot: On the fairness of detection and re-identification in multiple object tracking", Int. J. Comput. Vision, 129(11), 3069-3087. https://doi.org/10.1007/s11263-021-01513-4
  52. Zhou, X., Koltun, V. and Krahenbuhl, P. (2020), "Tracking objects as points", Proceedings of the European Conference on Computer Vision, Glasgow, UK, August.