Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

A completely non-contact recognition system for bridge unit influence line using portable cameras and computer vision

  • Dong, Chuan-Zhi (Department of Civil, Environmental, and Construction Engineering, University of Central Florida) ;
  • Bas, Selcuk (Department of Civil, Environmental, and Construction Engineering, University of Central Florida) ;
  • Catbas, F. Necati (Department of Civil, Environmental, and Construction Engineering, University of Central Florida)
  • Received : 2019.06.18
  • Accepted : 2019.08.25
  • Published : 2019.11.25
⟨ Previous Next ⟩

Abstract

Currently most of the vision-based structural identification research focus either on structural input (vehicle location) estimation or on structural output (structural displacement and strain responses) estimation. The structural condition assessment at global level just with the vision-based structural output cannot give a normalized response irrespective of the type and/or load configurations of the vehicles. Combining the vision-based structural input and the structural output from non-contact sensors overcomes the disadvantage given above, while reducing cost, time, labor force including cable wiring work. In conventional traffic monitoring, sometimes traffic closure is essential for bridge structures, which may cause other severe problems such as traffic jams and accidents. In this study, a completely non-contact structural identification system is proposed, and the system mainly targets the identification of bridge unit influence line (UIL) under operational traffic. Both the structural input (vehicle location information) and output (displacement responses) are obtained by only using cameras and computer vision techniques. Multiple cameras are synchronized by audio signal pattern recognition. The proposed system is verified with a laboratory experiment on a scaled bridge model under a small moving truck load and a field application on a footbridge on campus under a moving golf cart load. The UILs are successfully identified in both bridge cases. The pedestrian loads are also estimated with the extracted UIL and the predicted weights of pedestrians are observed to be in acceptable ranges.

Keywords

Acknowledgement

Supported by : NSF

References

  1. Canny, J.F. (1986), "A computational approach to edge detection", IEEE T. Pattern Anal. Mach. Intell., 8(6), 679-698. https://doi.org/10.1109/TPAMI.1986.4767851
  2. Catbas, N., Zaurin, R., Gul, M. and Gokce, H.B. (2012), "Sensor networks, computer imaging, and unit influence lines for structural health monitoring: case study for bridge load rating",J. Bridg. Eng., 17(4), 662-670. DOI: 10.1061/(ASCE)BE.1943-5592.0000288.
  3. Chen, S.-E., Liu, W., Bian, H. and Smith, B. (2012), "3D liDAR scans for bridge damage evaluations", Sixth Congr. Forensic Eng., San Francisco, California, United States, 487-495.
  4. Catbas, F.N., Hiasa, S., Dong, C., Pan, Y., Celik, O. and Karaaslan, E. (2017), "Comprehensive Structural Health Monitoring at Local and Global Level with Vision-based Technologies", 26th ASNT Res. Symp.
  5. Catbas, F.N., Dong, C.Z., Celik, O. and Khuc, T. (2018), "A vision for vision-based technologies for bridge health monitoring", Maintenance, Safety, Risk, Manag. Life-Cycle Perform. Bridg. Proc. Ninth Int. Conf. Bridg. Maintenance, Saf. Manag. (IABMAS 2018), Melbourne, Australia, 54.
  6. Celik, O., Dong, C.Z. and Catbas, F.N. (2018a), "A computer vision approach for the load time history estimation of lively individuals and crowds", Comput. Struct., 200, 32-52. DOI:10.1016/j.compstruc.2018.02.001.
  7. Celik, O., Terrell, T., Gul, M. and Catbas, F.N. (2018b), "Sensor clustering technique for practical structural monitoring and maintenance", Struct. Monit. Maint., 5(2), 273-295. DOI: 10.12989/smm.2018.5.2.273.
  8. Chen, J.G., Adams, T.M., Sun, H., Bell, E.S. and Buyukozturk, O. (2018a), "Camera-based vibration measurement of the world war I memorial bidge in portsmouth, new hampshire", J. Struct. Eng., 144(11), 04018207. DOI:10.1061/(ASCE)ST.1943-541X.0002203.
  9. Chen, Y., Joffre, D. and Avitabile, P. (2018b), "Underwater dynamic response at limited points expanded to full-field strain response", J. Vib. Acoust., 140(5), 051016. DOI:10.1115/1.4039800.
  10. Chen, Y., Logan, P., Avitabile, P. and Dodson, J. (2019), "Non-model based expansion from limited points to an augmented set of points using chebyshev polynomials", Exp. Tech., 1-23. DOI:10.1007/s40799-018-00300-0.
  11. Dong, C.Z., Ye, X.W. and Liu, T. (2015), "Non-contact structural vibration monitoring under varying environmental conditions", Vib. Pro., 5, 217-222.
  12. Duan, Y.F., Zhang, R., Dong, C.Z., Luo, Y. Z., Or, S.W., Zhao, Y. and Fan, K.-Q. (2015), "Development of elasto-magneto-Electric (EME) sensor for in-Service cable force monitoring", Int. J. Struct. Stab. Dyn., 16(04), 1640016. DOI:10.1142/S0219455416400162.
  13. Dong, C.Z., Celik, O. and Catbas, F.N. (2018a), "Marker free monitoring of the grandstand structures and modal identificationusing computer vision methods", Struct. Heal. Monit., 1-19. DOI:10.1177/1475921718806895.
  14. Dong, C.Z., Ye, X.W. and Jin, T. (2018b), "Identification of structural dynamic characteristics based on machine vision technology", Meas. J. Int. Meas. Confed., 126, 405-416. DOI: 10.1016/j.measurement.2017.09.043.
  15. Dong, C.Z. and Catbas, F.N. (2019), "A non-target structural displacement measurement method using advanced feature matching strategy", Adv. Struct. Eng. DOI: 10.1177/1369433219856171.
  16. Dong, C.Z., Celik, O., Catbas, F.N., O'Brien, E.J. and Taylor, S. (2019a), "Structural displacement monitoring using deep learning-based full field optical flow methods", Struct. Infrastruct. Eng., 1-21. DOI: 10.1080/15732479.2019.1650078.
  17. Dong, C.Z., Celik, O., Catbas, F.N., OBrien, E.J. and Taylor, S.E. (2019b), "A robust non-contact displacement measurement method for long-term bridge monitoring using spatio-temporal context learning and taylor approximation", Sensors, 19(14), 3197. https://doi.org/10.3390/s19143197
  18. Feng, D., Scarangello, T., Feng, M.Q. and Ye, Q. (2017), "Cable tension force estimate using novel noncontact vision-based sensor", Meas. J. Int. Meas. Confed., 99, 44-52. DOI:10.1016/j.measurement.2016.12.020.
  19. Girshick, R., Donahue, J., Darrell, T., Berkeley, U.C. and Malik, J. (2012),"(rcnn) Rich feature hierarchies for accurate object detection and semantic segmentation", 2014 IEEE Conf. Comput. Vis. Pattern Recognit., 2-9.
  20. Girshick, R. (2015), "Fast r-cnn", Int. Conf. Comput. Vis., 1440-1448. DOI:10.1109/ICCV.2015.169.
  21. Hartley, R. and Zisserman, A. (2003), "Multiple view geometry in computer vision", Cambridge University Press. DOI:10.1109/ICCV.2009.5459414.
  22. He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), "Mask r-cnn", IEEE Int. Conf. Comput. Vis., 2961-2969. DOI:10.1109/TPAMI.2018.2844175.
  23. Hiasa, S., Necati Catbas, F., Matsumoto, M. and Mitani, K. (2017), "Considerations and issues in the utilization of infrared thermography for concrete bridge inspection at normal driving speeds", J. Bridg. Eng., 22(11), 04017101. DOI: 10.1061/(ASCE)BE.1943-5592.0001124.
  24. Hoskere, V., Park, J., Yoon, H. and Spencer Jr., B.F. (2019), "Vision-based modal survey of civil infrastructure using unmanned aerial vehicles", J. Struct. Eng., 145(7), 04019062. DOI: 10.1061/(ASCE)ST.1943-541X.0002321.
  25. Khuc, T. and Catbas, F.N. (2016), "Computer vision-based displacement and vibration monitoring without using physical target on structures",Struct. Infrastruct. Eng., Taylor & Francis, 13(4), 1-12. DOI:10.1080/15732479.2016.1164729.
  26. Khuc, T. and Catbas, F.N. (2017), "Completely contactless structural health monitoring of real-life structures using cameras and computer vision",Struct. Control Health. Monit., 24(1), e1852. DOI:10.1002/stc.1852.
  27. Karaaslan, E., Bagci, U. and Catbas, F.N. (2018). "Artificial intelligence assisted infrastructure assessment using mixed reality systems", arXiv Prepr., arXiv:1812.05659.
  28. Khuc, T. and Catbas, F.N. (2018). "Structural identification using computer vision-based bridge health monitoring", J. Struct. Eng., 144(2), 04017202. https://doi.org/10.1061/(ASCE)ST.1943-541X.0001925
  29. Kong, X., Li, J., Collins, W., Bennett, C., Laflamme, S. and Jo, H. (2018), "Sensing distortion-induced fatigue cracks in steel bridges with capacitive skin sensor arrays", Smart Mater. Struct., 27(11), 115008. DOI: 10.1088/1361-665X/aadbfb.
  30. Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pfugfelder, R., Zajc, L.C., Vojir, T., Bhat, G., Lukezic, A., Eldesokey, A., Fernandez, G., et al. (2018), "The sixth visual object tracking VOT2018 challenge results", VOT2018 Work. ECCV2018, Munich, Germany.
  31. Lee, J.J., Cho, S. and Shinozuka, M. (2006), "Evaluation of bridge load carrying capacity based on dynamic displacement measurement using real-time image processing techniques", Steel Struct., 6, 377-385.
  32. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y. and Berg, A.C. (2016). "SSD: single shot multibox detector", Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 9905 LNCS, 21-37. DOI:10.1007/978-3-319-46448-0_2.
  33. Lukezic, A., Vojir, T., Zajc, L.C., Matas, J. and Kristan, M. (2017), "Discriminative correlation filter tracker with channel and spatial reliability", Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 6309-6318. DOI:10.1007/s11263-017-1061-3.
  34. Ni, Y.Q., Ye, X.W. and Ko, J.M. (2010), "Monitoring-based fatigue reliability assessment of steel bridges: analytical model and application", J. Struct. Eng. - ASCE, 136(12), 1563-1573. DOI:10.1061/(ASCE)ST.1943-541X.0000250.
  35. Ni, Y.Q., Ye, X.W. and Ko, J.M. (2011), "Modeling of stress spectrum using long-term monitoring data and finite mixture distributions", J. Eng. Mech. - ASCE, 138(2), 175-184. DOI: 10.1061/(ASCE)EM.1943-7889.0000313.
  36. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2015), "You only look once: unified, real-time object detection", Proc. 2016 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR 2016), 779-788. DOI:10.1109/CVPR.2016.91.
  37. Ren, S., He, K., Girshick, R. and Sun, J. (2017), "Faster r-cnn: towards real-time object detection with region proposal networks", IEEE T. Pattern Anal. Mach. Intell., 39(6), 1137-1149. DOI:10.1109/TPAMI.2016.2577031.
  38. Tian, Y., Zhang, J. and Yu, S. (2018), "Rapid impact testing and system identification of footbridges using particle image velocimetry", Comput. Civ. Infrastruct. Eng., 1-16. DOI: 10.1111/mice.12390.
  39. Tian, Y., Zhang, J. and Yu, S. (2019), "Vision-based structural scaling factor and flexibility identification through mobile impact testing", Mech. Syst. Signal Pr., 122, 387-402. DOI: 10.1016/j.ymssp.2018.12.029.
  40. Xu, Y. and Brownjohn, J.M.W. (2018), "Review of machine-vision based methodologies for displacement measurement in civil structures", J. Civ. Struct. Heal. Monit., Springer Berlin Heidelberg, 8(1), 91-110. DOI:10.1007/s13349-017-0261-4.
  41. Ye, X.W., Ni, Y.Q., Wai, T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013a), "A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification", Smart Struct. Syst., 12(3-4), 363-379. http://dx.doi.org/10.12989/sss.2013.12.3_4.363.
  42. Ye, X.W., Ni, Y.Q., Wong, K.Y. and Ko, J.M. (2013b), "Statistical analysis of stress spectra for fatigue life assessment of steel bridges with structural health monitoring data", Eng. Struct., 45, 166-176. DOI: 10.1016/j.engstruct.2012.06.016.
  43. Ye, X.W., Yi, T.-H., Dong, C.Z., Liu, T. and Bai, H. (2015a), "Multi-point displacement monitoring of bridges using a vision-based approach", Wind Struct., 20(2), 315-326. http://dx.doi.org/10.12989/was.2015.20.2.315.
  44. Ye, X.W., Yi, T.H., Wen, C. and Su, Y.H. (2015b), "Reliability-based assessment of steel bridge deck using a mesh-insensitive structural stress method", Smart Struct. Syst., 16(2), 367-382. http://dx.doi.org/10.12989/sss.2015.16.2.367.
  45. Ye, X.W., Dong, C.Z. and Liu, T. (2016a), "A review of machine vision-based structural health monitoring: methodologies and applications", Sensors, 2016. DOI: 10.1155/2016/7103039.
  46. Ye, X.W., Dong, C.Z. and Liu, T. (2016b), "Image-based structural dynamic displacement measurement using different multi-object tracking algorithms", Smart Struct. Syst., 17(6), 935-956. http://dx.doi.org/10.12989/sss.2016.17.6.935.
  47. Ye, X.W., Dong, C.Z. and Liu, T. (2016c), "Force monitoring of steel cables using vision-based sensing technology : methodology and experimental verification", Smart Struct. Syst., 18(3), 585-599. http://dx.doi.org/10.12989/sss.2016.18.3.585.
  48. Ye, X.W., Liu, T., Dong, C.Z. and Chen, B. (2016d), "FBG-based structural performance monitoring and safety evaluation of a composite arch bridge", Struct. Heal. Monit. Compos. Struct. Using Fiber Opt. Methods, CRC Press, 269-297.
  49. Ye, X.W., Su, Y.H., Xi, P.S., Chen, B. and Han, J.P. (2016e). "Statistical analysis and probabilistic modeling of WIM monitoring data of an instrumented arch bridge", Smart Struct. Syst., 17(6), 1087-1105. http://dx.doi.org/10.12989/sss.2016.17.6.1087.
  50. Ye, X.W., Yi, T.H., Dong, C.Z. and Liu, T. (2016f), "Vision-based structural displacement measurement: system performance evaluation and influence factor analysis", Meas. J. Int. Meas. Confed., 88, 372-384. DOI: 10.1016/j.measurement.2016.01.024.
  51. Yang, Y., Dorn, C., Mancini, T., Talken, Z., Nagarajaiah, S., Kenyon, G., Farrar, C. and Mascarenas, D. (2017), "Blind identification of full-field vibration modes of output-only structures from uniformly-sampled, possibly temporally-aliased (sub-Nyquist), video measurements", J. Sound Vib., 390, 232-256. DOI:10.1016/j.jsv.2016.11.034.
  52. Ye, X.W., Yi, T.H., Su, Y.H., Liu, T. and Chen, B. (2017), "Strain-based structural condition assessment of an instrumented arch bridge using FBG monitoring data", Smart Struct. Syst., 20(2), 139-150. http://dx.doi.org/10.12989/sss.2017.20.2.139.
  53. Zhang, Z. (2002). "A flexible new technique for camera calibration (technical report)", IEEE T. Pattern Anal. Mach. Intel., 22(11), 1330-1334. https://doi.org/10.1109/34.888718
  54. Zaurin, R., and Catbas, F. (2010a), "Structural health monitoring using video stream, influence lines, and statistical analysis", Struct. Health. Monit., 10(3), 309-332. DOI: 10.1177/1475921710373290.
  55. Zaurin, R. and Catbas, F. (2010b), "Integration of computer imaging and sensor data for structural health monitoring of bridges", Smart Mater. Struct., 19, 1-15. DOI:10.1088/0964-1726/19/1/015019.
  56. Zaurin, R., Khuc, T. and Catbas, F.N. (2015), "Hybrid sensor-camera monitoring for damage detection: case study of a real bridge", J. Bridg. Eng., 21(6), 1-27. DOI: 10.1061/(ASCE)BE.1943-5592.0000811.
  57. Zhong, F.Q., Indurkar, P.P. and Quan, C.G. (2018). "Three-dimensional digital image correlation with improved efficiency and accuracy", Meas. J. Int. Meas. Confed., 128, 23-33. DOI:10.1016/j.measurement.2018.06.022.
  58. Zhong, F., Kumar, R. and Quan, C. (2019a), "A cost-effective single-shot structured light system for 3D shape measurement", IEEE Sens. J., IEEE, 19(17), 7335-7346. DOI: 10.1109/JSEN.2019.2915986.
  59. Zhong, F. and Quan, C. (2018), "Efficient digital image correlation using gradient orientation", Opt. Laser Technol., 106, 417-426. DOI:10.1016/j.optlastec.2018.04.024.
  60. Zhong, F.Q., Shao, X.X. and Quan, C. (2019b), "A comparative study of 3D reconstruction methods in stereo digital image correlation", Opt. Lasers Eng., 122, 142-150. https://doi.org/10.1016/j.optlaseng.2019.06.001

Cited by

  1. Performance of Camera-Based Vibration Monitoring Systems in Input-Output Modal Identification Using Shaker Excitation vol.13, pp.17, 2019, https://doi.org/10.3390/rs13173471