Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Long-term shape sensing of bridge girders using automated ROI extraction of LiDAR point clouds

  • Ganesh Kolappan Geetha (Department of Mechanical Engineering, Indian Institute of Technology Bhilai) ;
  • Sahyeon Lee (Digital Convergence Research Division, Korea Expressway Corporation Research Institute) ;
  • Junhwa Lee (Department of Civil Engineering, Pukyong National University) ;
  • Sung-Han Sim (Department of Global Smart City, Sungkyunkwan University)
  • Received : 2023.11.08
  • Accepted : 2024.07.22
  • Published : 2024.06.25
⟨ Previous Next ⟩

Abstract

This study discusses the long-term deformation monitoring and shape sensing of bridge girder surfaces with an automated extraction scheme for point clouds in the Region Of Interest (ROI), invariant to the position of a Light Detection And Ranging system (LiDAR). Advanced smart construction necessitates continuous monitoring of the deformation and shape of bridge girders during the construction phase. An automated scheme is proposed for reconstructing geometric model of ROI in the presence of noisy non-stationary background. The proposed scheme involves (i) denoising irrelevant background point clouds using dimensions from the design model, (ii) extracting the outer boundaries of the bridge girder by transforming and processing the point cloud data in a two-dimensional image space, (iii) extracting topology of pre-defined targets using the modified Otsu method, (iv) registering the point clouds to a common reference frame or design coordinate using extracted predefined targets placed outside ROI, and (v) defining the bounding box in the point clouds using corresponding dimensional information of the bridge girder and abutments from the design model. The surface-fitted reconstructed geometric model in the ROI is superposed consistently over a long period to monitor bridge shape and derive deflection during the construction phase, which is highly correlated. The proposed scheme of combining 2D-3D with the design model overcomes the sensitivity of 3D point cloud registration to initial match, which often leads to a local extremum.

Keywords

Acknowledgement

The authors gratefully acknowledge financial support from a grant (NRF-2020R1A2C2014797) from the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education and National R&D Project for Smart Construction Technology (RS-2020-KA156887) funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure and Transport and managed by the Korea Expressway Corporation.

References

  1. Cabaleiro, M., Riveiro, B., Conde, B. and Sanchez-Rodriguez, A. (2020), "A case study of measurements of deformations due to different loads in pieces less than 1 m from lidar data", Measurement, 151, 107196. https://doi.org/10.1016/j.measurement.2019.107196 
  2. Cha, G., Park, S. and Oh, T. (2019), "A terrestrial LiDAR-based detection of shape deformation for maintenance of bridge structures", J. Constr. Eng. Manag., 145(12), 04019075. https://doi.org/10.1061/(ASCE)CO.1943-7862.00017 
  3. Cha, G., Sim, S. H., Park, S. and Oh, T. (2020), "LiDAR-based bridge displacement estimation using 3D spatial optimization", Sensors, 20(24), 7117. https://doi.org/10.3390/s20247117 
  4. Cho, S., Lee, J. and Sim, S.H. (2018), "Comparative study on displacement measurement sensors for high-speed railroad bridge", Smart Struct. Syst., Int. J., 21(5), 637-652. https://doi.org/10.12989/sss.2018.21.5.637 
  5. Colombani, I.A. and Andrawes, B. (2022), "A study of multi-target image-based displacement measurement approach for field testing of bridges", J. Struct. Integr. Maint., 7(4), 207-216. https://doi.org/10.1080/24705314.2022.2088071 
  6. Di-Stefano, F., Chiappini, S., Gorreja, A., Balestra, M. and Pierdicca, R. (2021), "Mobile 3D scan LiDAR: A literature review", Geomat. Natural Hazards Risk, 12(1), 2387-2429. https://doi.org/10.1080/19475705.2021.1964617 
  7. Durrett, R. (2019), Probability: Theory and Examples, Vol. 49, Cambridge University 
  8. Gharehbaghi, V.R., Kalbkhani, H., Noroozinejad Farsangi, E., Yang, T.Y., Nguyen, A., Mirjalili, S. and Malaga-Chuquitaype, C. (2022), "A novel approach for deterioration and damage identification in building structures based on Stockwell-Transform and deep convolutional neural network", J. Struct. Integr. Maint., 7(2), 136-150. https://doi.org/10.1080/24705314.2021.2018840 
  9. Girardeau-Montaut, D., Roux, M., Marc, R. and Thibault, G. (2005), "Change detection on points cloud data acquired with a ground laser scanner. International Archives of Photogrammetry", Remote Sens. Spatial Inform. Sci., 36(3). 
  10. Gordon, S.J. and Lichti, D.D. (2007), "Modeling terrestrial laser scanner data for precise structural deformation measurement", J. Survey. Eng., 133(2), 72-80. https://doi.org/10.1061/(ASCE)0733-9453(2007)133:2(72) 
  11. Graves, W., Aminfar, K. and Lattanzi, D. (2022), "Full-scale highway bridge deformation tracking via photogrammetry and remote sensing", Remote Sensing, 14(12), 2767. https://doi.org/10.3390/rs14122767 
  12. Hough, P.V. (1962), U.S. Patent No. 3,069,654, Washington, DC: U.S. Patent and Trademark Office. 
  13. Hu, F., Zhao, J., Huang, Y. and Li, H. (2021), "Structure-aware 3D reconstruction for cable-stayed bridges: A learning-based method", Comput.-Aided Civil Infrastr. Eng., 36(1), 89-108. https://doi.org/10.1111/mice.12568 
  14. Kim, M.K., Sohn, H. and Chang, C.C. (2015), "Localization and quantification of concrete spalling defects using terrestrial laser scanning", J. Comput. Civil Eng., 29(6), 04014086. https://doi.org/10.1061/(ASCE)CP.1943-5487.00004 
  15. Kim, M.K., Wang, Q., Park, J.W., Cheng, J.C., Sohn, H. and Chang, C.C. (2016), "Automated dimensional quality assurance of full-scale precast concrete elements using laser scanning and BIM", Automat. Constr., 72, 102-114. https://doi.org/10.1016/j.autcon.2016.08.035 
  16. Kim, H., Yoon, J. and Sim, S.H. (2020), "Automated bridge component recognition from point clouds using deep learning", Struct. Control Health Monitor., 27(9), e2591. https://doi.org/10.1002/stc.2591 
  17. Kim, H., Lee, S., Ahn, E., Shin, M. and Sim, S.H. (2021a), "Crack identification method for concrete structures considering angle of view using RGB-D camera-based sensor fusion", Struct. Health Monitor., 20(2), 500-512. https://doi.org/10.1177/1475921720934758 
  18. Kim, H., Yoon, J., Hong, J. and Sim, S.H. (2021b), "Automated damage localization and quantification in concrete bridges using point cloud-based surface-fitting strategy", J. Comput. Civil Eng., 35(6), 04021028. https://doi.org/10.1061/(ASCE)CP.1943-5487.00009 
  19. Kolappan Geetha, K. and Sim, S.H. (2022), "Fast identification of concrete cracks using 1D deep learning and explainable artificial intelligence-based analysis", Automat. Constr., 143, 104572. https://doi.org/10.1016/j.autcon.2022.104572 
  20. Kolappan Geetha, G., Yang, H.J. and Sim, S.H. (2023), "Fast detection of missing thin propagating cracks during deep-learning-based concrete crack/non-crack classification", Sensors, 23(3), 1419, https://doi.org/10.3390/s23031419 
  21. Lague, D., Brodu, N. and Leroux, J. (2013), "Accurate 3D comparison of complex topography with terrestrial laser scanner: Application to the Rangitikei canyon (NZ)", ISPRS J. Photogramm. Remote Sens., 82, 10-26. https://doi.org/10.1016/j.isprsjprs.2013.04.009 
  22. Lee, J., Lee, K.C., Cho, S. and Sim, S.H. (2017), "Computer vision-based structural displacement measurement robust to light-induced image degradation for in-service bridges", Sensors, 17(10), 2317. https://doi.org/10.3390/s17102317 
  23. Lee, J., Lee, K.C., Lee, S., Lee, Y.J. and Sim, S.H. (2019), "Long-term displacement measurement of bridges using a LiDAR system", Struct. Control Health Monitor., 26(10), e2428. https://doi.org/10.1002/stc.2428 
  24. Lee, J., Lee, K.C., Jeong, S., Lee, Y.J. and Sim, S.H. (2020a), "Long-term displacement measurement of full-scale bridges using camera ego-motion compensation", Mech. Syst. Signal Process., 140, 106651. https://doi.org/10.1016/j.ymssp.2020.106651 
  25. Lee, S., Lee, J., Park, J.W. and Sim, S.H. (2020b), "Nontarget-based measurement of 6-DOF structural displacement using combined RGB color and depth information", IEEE/ASME Transact. Mechatron., 26(3), 1358-1368. https://doi.org/10.1109/TMECH.2020.3019288 
  26. Lee, J.S., Park, J. and Ryu, Y.M. (2021), "Semantic segmentation of bridge components based on hierarchical point cloud model", Automat. Constr., 130, 103847. https://doi.org/10.1016/j.autcon.2021.103847 
  27. Li, Y., Ma, L. Zhong, Z., Liu, F., Cao, D., Li, J. and Chapman, M.A. (2020), "Deep learning for lidar point clouds in autonomous driving: A review", IEEE Trans. Neural Networks Learn. Syst., 32(8), 3412-3432. https://doi.org/10.48550/arXiv.2005.09830 
  28. Liu, M., Sun, X., Wang, Y., Shao, Y. and You, Y. (2020), "Deformation measurement of highway bridge head based on mobile TLS data", IEEE Access, 8, 85605-85615. https://doi.org/10.1109/ACCESS.2020.2992590 
  29. Lu, R., Brilakis, I. and Middleton, C.R. (2019), "Detection of structural components in point clouds of existing RC bridges", Comput.-Aided Civil Infrastr. Eng., 34(3), 191-212. https://doi.org/10.1111/mice.12407 
  30. Melo, A.G., Pinto, M.F., Honorio, L.M., Dias, F.M. and Masson, J.E. (2020), "3D correspondence and point projection method for structures deformation analysis", IEEE Access, 8, 177823-177836. 10.1109/ACCESS.2020.3027205 
  31. Napolitano, R., Liu, Z., Sun, C. and Glisic, B. (2019), "Combination of image-based documentation and augmented reality for structural health monitoring and building pathology", Front. Built Environ., 5, 50. https://doi.org/10.3389/fbuil.2019.00050 
  32. Ng, H.F., Jargalsaikhan, D., Tsai, H.C. and Lin, C.Y. (2013), "An improved method for image thresholding based on the valley-emphasis method", In: Asia-Pacific Signal and Information Processing Association Annual Summit and Conference IEEE, pp. 1-4. https://doi.org/10.1109/APSIPA.2013.6694261 
  33. Nocerino, E., Stathopoulou, E.K., Rigon, S. and Remondino, F. (2020), "Surface reconstruction assessment in photogrammetric applications", Sensors, 20(20), 5863. https://doi.org/10.3390/s20205863 
  34. Oskouie, P., Becerik-Gerber, B. and Soibelman, L. (2016), "Automated measurement of highway retaining wall displacements using terrestrial laser scanners", Automat. Constr., 65, 86-101. https://doi.org/10.1016/j.autcon.2015.12.023 
  35. Otsu, N. (1979), "A threshold selection method from gray-level histograms", IEEE Transact. Syst. Man Cybernet., 9(1), 62-66. https://doi.org/10.1109/TSMC.1979.4310076 
  36. Pan, Y., Dong, Y., Wang, D., Chen, A. and Ye, Z. (2019), "Three-dimensional reconstruction of structural surface model of heritage bridges using UAV-based photogrammetric point clouds", Remote Sensing, 11(10), 1204. https://doi.org/10.3390/rs11101204 
  37. Park, H.S., Lee, H.M., Adeli, H. and Lee, I. (2007), "A new approach for health monitoring of structures: terrestrial laser scanning", Comput.-Aided Civil Infrastr. Eng., 22(1), 19-30. https://doi.org/10.1111/j.1467-8667.2006.00466.x 
  38. Royo, S. and Ballesta-Garcia, M. (2019), "An overview of lidar imaging systems for autonomous vehicles", Appl. Sci., 9(19), 4093. https://doi.org/10.3390/app9194093 
  39. Rutzinger, M., Rottensteiner, F. and Pfeifer, N. (2009), "A comparison of evaluation techniques for building extraction from airborne laser scanning", IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, 2(1), 11-20. https://doi.org/10.1109/JSTARS.2009.2012488 
  40. Schnabel, R., Wahl, R. and Klein, R. (2007), "Efficient RANSAC for point-cloud shape detection", In: Computer Graphics Forum, Oxford, UK: Blackwell Publishing Ltd., 26(2), pp. 214-226. https://doi.org/10.1111/j.1467-8659.2007.01016.x 
  41. Sohn, H. and Park, B. (2021), "Laser-based structural health monitoring", In: Beer, M., Kougioumtzoglou, I., Patelli, E., Au, IK. (eds) Encyclopedia of Earthquake Engineering, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36197-5_86-1 
  42. Soudarissanane, S., Lindenbergh, R., Menenti, M. and Teunissen, P. (2011), "Scanning geometry: Influencing factor on the quality of terrestrial laser scanning points", ISPRS J. Photogramm. Remote Sens., 66(4), 389-399. https://doi.org/10.1016/j.isprsjprs.2011.01.005 
  43. Strang, G. (2022), Introduction to Linear Algebra, Wellesley-Cambridge Press.
  44. Tan, K. and Cheng, X. (2016a), "Correction of incidence angle and distance effects on TLS intensity data based on reference targets", Remote Sensing, 8(3), 251. https://doi.org/10.3390/rs8030251 
  45. Tan, K. and Cheng, X. (2016b), "Surface reflectance retrieval from the intensity data of a terrestrial laser scanner", JOSA A, 33(4), 771-778. https://doi.org/10.1364/JOSAA.33.000771 
  46. Tan, K., Chen, J., Qian, W., Zhang, W., Shen, F. and Cheng, X. (2019), "Intensity data correction for long-range terrestrial laser scanners: A case study of target differentiation in an intertidal zone", Remote Sensing, 11(3), 331. https://doi.org/10.3390/rs11030331 
  47. Torr, P.H. and Zisserman, A. (2000), "MLESAC: A new robust estimator with application to estimating image geometry", Comput. Vision Image Understand., 78(1), 138-156. https://doi.org/10.1006/cviu.1999.0832 
  48. Truong, M.T.N. and Kim, S (2018), "Automatic image thresholding using Otsu's method and entropy weighting scheme for surface defect detection", Soft Computat., 22, 4197-4203. https://doi.org/10.1007/s00500-017-2709-1 
  49. Truong-Hong, L. and Lindenbergh, R. (2019), "Measuring deformation of bridge structures using laser scanning data", In: 4th Joint International Symposium on Deformation Monitoring (JISDM), p. 7. 
  50. Truong-Hong, L. and Lindenbergh, R. (2022), "Automatically extracting surfaces of reinforced concrete bridges from terrestrial laser scanning point clouds", Automat. Constr., 135, 104127. https://doi.org/10.1016/j.autcon.2021.104127 
  51. Truong-Hong, L., Lindenbergh, R. and Nguyen, T.A. (2021), "Structural assessment using terrestrial laser scanning point clouds", Int. J. Build. Pathol. Adaptat., 40(3), 345-379.  https://doi.org/10.1108/IJBPA-04-2021-0051
  52. Wang, Z. and Menenti, M. (2021), "Challenges and opportunities in Lidar remote sensing", Front. Remote Sens., 2, p. 3. https://doi.org/10.3389/frsen.2021.641723 
  53. Wang, K., Ma, S., Chen, J., Ren, F. and Lu, J. (2020), "Approaches, challenges, and applications for deep visual odometry: Toward complicated and emerging areas", IEEE Transact. Cognit. Develop. Syst., 14(1), 35-49. https://doi.org/10.1109/TCDS.2020.3038898 
  54. Xia, T., Yang, J. and Chen, L. (2022), "Automated semantic segmentation of bridge point cloud based on local descriptor and machine learning", Automat. Constr., 133, 103992. https://doi.org/10.1016/j.autcon.2021.103992 
  55. Xu, Y. and Stilla, U. (2021), "Towards building and civil infrastructure reconstruction from point clouds: A review on data and key techniques", IEEE J. Select. Topics Appl. Earth Observ. Remote Sens., 14, 2857-2885. https://doi.org/10.1109/JSTARS.2021.3060568 
  56. Yan, Y. and Hajjar, J.F. (2021), "Automated extraction of structural elements in steel girder bridges from laser point clouds", Automat. Constr., 125, 103582. https://doi.org/10.1016/j.autcon.2021.103582 
  57. Yang, T., Li, Y., Zhao, C., Yao, D., Chen, G., Sun, L., Krajnik, and T. and Yanet, Z. (2022), "3D ToF LiDAR in mobile robotics: A review", arXiv Prepr, arXiv2202.11025. https://doi.org/10.48550/arXiv.2202.11025 
  58. Ye, C., Acikgoz, S., Pendrigh, S., Riley, E. and DeJong, M.J. (2018), "Mapping deformations and inferring movements of masonry arch bridges using point cloud data", Eng. Struct., 173, 530-545. https://doi.org/10.1016/j.engstruct.2018.06.094 
  59. Ziolkowski, P., Szulwic, J. and Miskiewicz, M. (2018), "Deformation analysis of a composite bridge during proof loading using point cloud processing", Sensors, 18(12), 4332. https://doi.org/10.3390/s18124332