Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Computer vision-based remote displacement monitoring system for in-situ bridge bearings robust to large displacement induced by temperature change

  • Kim, Byunghyun (Department of Civil Engineering, University of Seoul) ;
  • Lee, Junhwa (Department of Urban and Environmental Engineering, Ulsan National Institute of Science and Technology (UNIST)) ;
  • Sim, Sung-Han (School of Civil, Architectural Engineering and Landscape Architecture, Sungkyunkwan University) ;
  • Cho, Soojin (Department of Civil Engineering, University of Seoul) ;
  • Park, Byung Ho (Technology Innovation Center, Seoul Facilities Corporation)
  • Received : 2021.11.10
  • Accepted : 2022.10.27
  • Published : 2022.11.25
⟨ Previous Next ⟩

Abstract

Efficient management of deteriorating civil infrastructure is one of the most important research topics in many developed countries. In particular, the remote displacement measurement of bridges using linear variable differential transformers, global positioning systems, laser Doppler vibrometers, and computer vision technologies has been attempted extensively. This paper proposes a remote displacement measurement system using closed-circuit televisions (CCTVs) and a computer-vision-based method for in-situ bridge bearings having relatively large displacement due to temperature change in long term. The hardware of the system is composed of a reference target for displacement measurement, a CCTV to capture target images, a gateway to transmit images via a mobile network, and a central server to store and process transmitted images. The usage of CCTV capable of night vision capture and wireless data communication enable long-term 24-hour monitoring on wide range of bridge area. The computer vision algorithm to estimate displacement from the images involves image preprocessing for enhancing the circular features of the target, circular Hough transformation for detecting circles on the target in the whole field-of-view (FOV), and homography transformation for converting the movement of the target in the images into an actual expansion displacement. The simple target design and robust circle detection algorithm help to measure displacement using target images where the targets are far apart from each other. The proposed system is installed at the Tancheon Overpass located in Seoul, and field experiments are performed to evaluate the accuracy of circle detection and displacement measurements. The circle detection accuracy is evaluated using 28,542 images captured from 71 CCTVs installed at the testbed, and only 48 images (0.168%) fail to detect the circles on the target because of subpar imaging conditions. The accuracy of displacement measurement is evaluated using images captured for 17 days from three CCTVs; the average and root-mean-square errors are 0.10 and 0.131 mm, respectively, compared with a similar displacement measurement. The long-term operation of the system, as evaluated using 8-month data, shows high accuracy and stability of the proposed system.

Keywords

Acknowledgement

This study was supported by the Korea Agency for Infrastructure Technology Advancement (KAIA), grant funded by the Ministry of Land, Infrastructure, and Transport (Grant 21CTAP-C163726-01).

References

  1. Atherton, T.J. and Kerbyson, D.J. (1999), "Size invariant circle detection", Image Vision Comput., 17(11), 795-803. https://doi.org/10.1016/s0262-8856(98)00160-7
  2. Casciati, F. and Fuggini, C. (2011), "Monitoring a steel building using GPS sensors", Smart Struct. Syst., Int. J., 7(5), 349-363. https://doi.org/10.12989/SSS.2011.7.5.349
  3. Cho, S., Park, J.W., Palanisamy, R.P. and Sim, S.H. (2016), "Reference-free displacement estimation of bridges using Kalman filter-based multimetric data fusion", J. Sensors, 2016. https://doi.org/10.1155/2016/3791856
  4. Dong, C.Z., Celik, O. and Catbas, F.N. (2019), "Marker-free monitoring of the grandstand structures and modal identification using computer vision methods", Struct. Health Monitor., 18(5-6), 1491-1509. https://doi.org/10.1177/1475921718806895/ASSET/IMAGES/LARGE/10.1177_1475921718806895-FIG2.JPEG
  5. Dworakowski, Z., Kohut, P., Gallina, A., Holak, K. and Uhl, T. (2016), "Vision-based algorithms for damage detection and localization in structural health monitoring", Struct. Control Health Monitor., 23(1), 35-50. https://doi.org/10.1002/stc.1755
  6. Federal Highway Administration (2007), Load and Resistance Factor Design (LRFD) for Highway Bridge Superstructures REFERENCE MANUAL.
  7. Feng, D. and Feng, M.Q. (2016), "Vision-based multipoint displacement measurement for structural health monitoring", Struct. Control Health Monitor., 23(5), 876-890. https://doi.org/10.1002/stc.1819
  8. Feng, M.Q., Fukuda, Y., Feng, D. and Mizuta, M. (2015), "Nontarget Vision Sensor for Remote Measurement of Bridge Dynamic Response", J. Bridge Eng., 20(12), 04015023. https://doi.org/10.1061/(asce)be.1943-5592.0000747
  9. Find circles using circular Hough transform - MATLAB imfindcircles (n.d.), Retrieved May 31, 2021, from https://www.mathworks.com/help/images/ref/imfindcircles.html
  10. Fukuda, Y., Feng, M.Q. and Shinozuka, M. (2010), "Cost-effective vision-based system for monitoring dynamic response of civil engineering structures", Struct. Control Health Monitor., 17(8), 918-936. https://doi.org/10.1002/stc.360
  11. Garcia-Sanchez, D., Fernandez-Navamuel, A., Sanchez, D.Z., Alvear, D. and Pardo, D. (2020), "Bearing assessment tool for longitudinal bridge performance", J. Civil Struct. Health Monitor., 10, 1023-1036. https://doi.org/10.1007/s13349-020-00432-1
  12. Garg, P., Moreu, F., Ozdagli, A., Taha, M.R. and Mascarenas, D. (2019), "Noncontact dynamic displacement measurement of structures using a moving laser Doppler vibrometer", J. Bridge Eng., 24(9), 04019089. https://doi.org/10.1061/(asce)be.1943-5592.0001472
  13. Garg, P., Nasimi, R., Ozdagli, A., Zhang, S., Mascarenas, D.D.L., Reda Taha, M. and Moreu, F. (2020), "Measuring transverse displacements using unmanned aerial systems laser doppler vibrometer (UAS-LDV): Development and field validation", Sensors, 20(21), 1-16. https://doi.org/10.3390/s20216051
  14. Guo, T., Liu, J., Zhang, Y. and Pan, S. (2015), "Displacement Monitoring and Analysis of Expansion Joints of Long-Span Steel Bridges with Viscous Dampers", J. Bridge Eng., 20(9), 04014099. https://doi.org/10.1061/(ASCE)BE.1943
  15. Hoskere, V., Asce, S.M., Park, J.-W., Yoon, H., Asce, A.M., Spencer, B.F. and Asce, F. (2019), "Vision-Based Modal Survey of Civil Infrastructure Using Unmanned Aerial Vehicles", J. Struct. Eng., 145(7), 04019062. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002321
  16. Huang, H.-B., Yi, T.-H., Li, H.-N. and Liu, H. (2018), "New Representative Temperature for Performance Alarming of Bridge Expansion Joints through Temperature-Displacement Relationship", J. Bridge Eng., 23(7), 04018043. https://doi.org/10.1061/(asce)be.1943-5592.0001258
  17. Hussan, M., Kaloop, M.R., Sharmin, F. and Kim, D. (2018), "GPS Performance Assessment of Cable-Stayed Bridge using Wavelet Transform and Monte-Carlo Techniques", KSCE J. Civil Eng., 22(11), 4385-4398. https://doi.org/10.1007/s12205-018-0438-3
  18. Jeong, Y., Park, D. and Park, K.H. (2017), "PTZ camera-based displacement sensor system with perspective distortion correction unit for early detection of building destruction", Sensors, 17(3), 430. https://doi.org/10.3390/s17030430
  19. Jo, H., Sim, S.H., Tatkowski, A., Spencer, B.F. and Nelson, M.E. (2013), "Feasibility of displacement monitoring using low-cost GPS receivers", Struct. Control Health Monitor., 20(9), 1240-1254. https://doi.org/10.1002/stc.1532
  20. Kanopoulos, N., Vasanthavada, N. and Baker, R.L. (1988), "Design of an Image Edge Detection Filter Using the Sobel Operator", IEEE J. Solid-State Circuits, 23(2), 358-367. https://doi.org/10.1109/4.996
  21. Kim, K., Choi, J., Chung, J., Koo, G., Bae, I.H. and Sohn, H. (2018), "Structural displacement estimation through multi-rate fusion of accelerometer and RTK-GPS displacement and velocity measurements", Measurement: J. Int. Measure. Confed., 130, 223-235. https://doi.org/10.1016/j.measurement.2018.07.090
  22. Korean Authority of Land and Infrastructure Safety (2019), Detailed guidelines for safety and maintenance of infrastructures (Bridges).
  23. Lee, J.J. and Shinozuka, M. (2006), "A vision-based system for remote sensing of bridge displacement", NDT and E Int., 39(5), 425-431. https://doi.org/10.1016/j.ndteint.2005.12.003
  24. Lee, J.J., Fukuda, Y., Shinozuka, M., Cho, S. and Yun, C.-B. (2007), "Development and application of a vision-based displacement measurement system for structural health monitoring of civil structures", Smart Struct. Syst., Int. J., 3(3), 373-384. https://doi.org/10.12989/sss.2007.3.3.373
  25. Lee, J., Lee, K.C., Cho, S. and Sim, S.H. (2017), "Computer vision-based structural displacement measurement robust to light-induced image degradation for in-service bridges", Sensors, 17(10), 2317. https://doi.org/10.3390/s17102317
  26. Lee, J., Lee, K.C., Jeong, S., Lee, Y.J. and Sim, S.H. (2020), "Long-term displacement measurement of full-scale bridges using camera ego-motion compensation", Mech. Syst. Signal Process., 140, 106651. https://doi.org/10.1016/J.YMSSP.2020.106651
  27. Luo, L., Feng, M.Q., Wu, J. and Bi, L. (2021), "Modeling and detection of heat haze in computer vision based displacement measurement", Measurement, 182, 109772. https://doi.org/10.1016/J.MEASUREMENT.2021.109772
  28. Ma, Z., Choi, J. and Sohn, H. (2022), "Real-time structural displacement estimation by fusing asynchronous acceleration and computer vision measurements", Comput.-Aided Civil Infrastr. Eng., 37(6), 688-703. https://doi.org/10.1111/MICE.12767
  29. Moschas, F., Psimoulis, P.A. and Stiros, S.C. (2013), "GPS/RTS data fusion to overcome signal deficiencies in certain bridge dynamic monitoring projects", Smart Struct. Syst., Int. J., 12(4), 1738-1991. https://doi.org/10.12989/sss.2013.12.3_4.251
  30. Nassif, H.H., Gindy, M. and Davis, J. (2005), "Comparison of laser Doppler vibrometer with contact sensors for monitoring bridge deflection and vibration", NDT and E Int., 38(3), 213-218. https://doi.org/10.1016/j.ndteint.2004.06.012
  31. Park, C. and Lee, H. (2016), "Prediction on domestic transportation infrastructure maintenance investment", Construction Economy Research Institute of Korea. http://www.cerik.re.kr/report/issue/detail/1964
  32. Park, J.W., Lee, J.J., Jung, H.J. and Myung, H. (2010), "Visionbased displacement measurement method for high-rise building structures using partitioning approach", NDT & E Int., 43(7), 642-647. https://doi.org/10.1016/J.NDTEINT.2010.06.009
  33. Ribeiro, D., Santos, R., Cabral, R., Saramago, G., Montenegro, P., Carvalho, H., Correia, J. and Calcada, R. (2021), "Non-contact structural displacement measurement using Unmanned Aerial Vehicles and video-based systems", Mech. Syst. Signal Process., 160, 107869. https://doi.org/10.1016/J.YMSSP.2021.107869
  34. Shao, Y., Li, L., Li, J., An, S. and Hao, H. (2021), "Computer vision based target-free 3D vibration displacement measurement of structures", Eng. Struct., 246, 113040. https://doi.org/10.1016/J.ENGSTRUCT.2021.113040
  35. Shariati, A., Schumacher, T. and Ramanna, N. (2015), "Eulerianbased virtual visual sensors to detect natural frequencies of structures", J. Civil Struct. Health Monitor., 5(4), 457-468. https://doi.org/10.1007/s13349-015-0128-5
  36. Shrestha, A., Dang, J., Nakajima, K. and Wang, X. (2020), "Image processing-based real-time displacement monitoring methods using smart devices", Struct. Control Health Monitor., 27(2), e2473. https://doi.org/10.1002/STC.2473
  37. Sladek, J., Ostrowska, K., Kohut, P., Holak, K., Gaska, A. and Uhl, T. (2013), "Development of a vision based deflection measurement system and its accuracy assessment", Measurement: J. Int. Measure. Confed., 46(3), 1237-1249. https://doi.org/10.1016/j.measurement.2012.10.021
  38. Song, Q., Wu, J., Wang, H., An, Y. and Tang, G. (2022), "Computer vision-based illumination-robust and multi-point simultaneous structural displacement measuring method", Mech. Syst. Signal Process., 170, 108822. https://doi.org/10.1016/J.YMSSP.2022.108822
  39. Watson, C., Watson, T. and Coleman, R. (2007), "Structural Monitoring of Cable-Stayed Bridge: Analysis of GPS versus Modeled Deflections", J. Survey. Eng., 133(1), 23-28. https://doi.org/10.1061/(asce)0733-9453(2007)133:1(23)
  40. Weng, Y., Shan, J., Lu, Z., Lu, X. and Spencer, B.F. (2021), "Homography-based structural displacement measurement for large structures using unmanned aerial vehicles", Comput.-Aided Civil Infrastr. Eng., 36(9), 1114-1128. https://doi.org/10.1111/MICE.12645
  41. Wu, L.J., Casciati, F. and Casciati, S. (2014), "Dynamic testing of a laboratory model via vision-based sensing", Eng. Struct., 60, 113-125. https://doi.org/10.1016/j.engstruct.2013.12.002
  42. Xia, Q., Zhou, L. and Zhang, J. (2018), "Thermal performance analysis of a long-span suspension bridge with long-term monitoring data", J. Civil Struct. Health Monitor., 8(4), 543- 553. https://doi.org/10.1007/s13349-018-0299-y
  43. Xing, L., Dai, W. and Zhang, Y. (2022), "Improving displacement measurement accuracy by compensating for camera motion and thermal effect on camera sensor", Mech. Syst. Signal Process., 167, 108525. https://doi.org/10.1016/J.YMSSP.2021.108525
  44. Xu, Y., Brownjohn, J.M.W., Hester, D. and Koo, K.Y. (2017), "Long-span bridges: Enhanced data fusion of GPS displacement and deck accelerations", Eng. Struct., 147, 639-651. https://doi.org/10.1016/j.engstruct.2017.06.018
  45. Ye, X.W., Ni, Y.Q., Wai, T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013), "A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification", Smart Struct. Syst., Int. J., 12(3_4), 363-379. https://doi.org/10.12989/SSS.2013.12.3_4.363
  46. Ye, X.W., Dong, C.Z. and Liu, T. (2016a), "Image-based structural dynamic displacement measurement using different multi-object tracking algorithms", Smart Struct. Syst., Int. J., 17(6), 935-956. https://doi.org/10.12989/sss.2016.17.6.935
  47. Ye, X.W., Yi, T.H., Dong, C.Z. and Liu, T. (2016b), "Vision-based structural displacement measurement: System performance evaluation and influence factor analysis", Measurement: J. Int. Measure. Confed., 88, 372-384. https://doi.org/10.1016/j.measurement.2016.01.024
  48. Yi, T.H., Li, H.N. and Gu, M. (2013), "Wavelet based multi-step filtering method for bridge health monitoring using GPS and accelerometer", Smart Struct. Syst., Int. J., 11(4), 331-348. https://doi.org/10.12989/SSS.2013.11.4.331
  49. Yoon, H., Elanwar, H., Choi, H., Golparvar-Fard, M. and Spencer, B.F. (2016), "Target-free approach for vision-based structural system identification using consumer-grade cameras", Struct. Control Health Monitor., 23(12), 1405-1416. https://doi.org/10.1002/STC.1850
  50. Zhao, H., Ding, Y., Nagarajaiah, S. and Li, A. (2019), "Longitudinal displacement behavior and girder end reliability of a jointless steel-truss arch railway bridge during operation", Appl. Sci., 9(11), 2222. https://doi.org/10.3390/app9112222
  51. Zhu, J., Zhang, C., Lu, Z. and Li, X. (2021), "A multi-resolution deep feature framework for dynamic displacement measurement of bridges using vision-based tracking system", Measurement, 183, 109847. https://doi.org/10.1016/J.MEASUREMENT.2021.109847