Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

BIM model-based structural damage localization using visual-inertial odometry

  • Junyeon Chung (Department of Civil Engineering, Korean Advanced Institute for Science and Technology) ;
  • Kiyoung Kim (Department of Civil Engineering, Korean Advanced Institute for Science and Technology) ;
  • Hoon Sohn (Department of Civil Engineering, Korean Advanced Institute for Science and Technology)
  • Received : 2023.05.02
  • Accepted : 2023.05.26
  • Published : 2023.06.25
⟨ Previous Next ⟩

Abstract

Ensuring the safety of a structure necessitates that repairs are carried out based on accurate inspections and records of damage information. Traditional methods of recording damage rely on individual paper-based documents, making it challenging for inspectors to accurately record damage locations and track chronological changes. Recent research has suggested the adoption of building information modeling (BIM) to record detailed damage information; however, localizing damages on a BIM model can be time-consuming. To overcome this limitation, this study proposes a method to automatically localize damages on a BIM model in real-time, utilizing consecutive images and measurements from an inertial measurement unit in close proximity to damages. The proposed method employs a visual-inertial odometry algorithm to estimate the camera pose, detect damages, and compute the damage location in the coordinate of a prebuilt BIM model. The feasibility and effectiveness of the proposed method were validated through an experiment conducted on a campus building. Results revealed that the proposed method successfully localized damages on the BIM model in real-time, with a root mean square error of 6.6 cm.

Keywords

Acknowledgement

This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (grant number 2022R1C1C2008186). This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-00075, Artificial Intelligence Graduate School Program (KAIST)).

References

  1. Agarwal, S., Snavely, N., Seitz, S.M. and Szeliski, R. (2010), "Bundle adjustment in the large", In: Computer Vision-ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September, Vol. 6312, pp. 29-42. https://doi.org/10.1007/978-3-642-15552-9_3
  2. American Society of Civil Engineers (2021), A Comprehensive Assessment of America's Infrastructure, ASCE, p. 111.
  3. Bazzucchi, F., Restuccia, L. and Ferro, G.A. (2018), "Considerations over the Italian road bridge infrastructure safety after the polcevera viaduct collapse: Past errors and future perspectives", Frat. ed Integrita Strutt., 12(46), 400-421. https://doi.org/10.3221/IGF-ESIS.46.37
  4. Besl, P. and McKay, N.D. (1992), "A method for registration of 3-D shapes", IEEE Trans. Pattern Anal. Mach. Intell., 14(2), 239-256. https://doi.org/10.1109/34.121791
  5. Biezma, M.V. and Schanack, F. (2007), "Collapse of steel bridges", J. Perform. Constr. Facil., 21(5), 398-405. https://doi.org/10.1061/(asce)0887-3828(2007)21:5(398)
  6. Chen, J., Li, S. and Lu, W. (2022), "Align to locate: Registering photogrammetric point clouds to BIM for robust indoor localization", Build. Environ., 209, 1-29. https://doi.org/10.1016/j.buildenv.2021.108675
  7. Dang, N., Kang, H., Lon, S. and Shim, C. (2018), "3D digital twin models for bridge maintenance", Proceedings of the 10th International Conference on Short Medium Span Bridge, Vol. 31, No. 73, pp. 1-9. [Online]. Available: https://www.researchgate.net/publication/331314334%0Ahttps://www.csce.ca/elf/apps/CONFERENCEVIEWER/conferences/SMSB/papers/FinalPaper_73_0508011616.doc
  8. Fisher, A., Cannizzaro, R., Cochrane, M., Nagahawatte, C. and Palmer, J.L. (2021), "ColMap : A memory-efficient occupancy grid mapping framework", Rob. Auton. Syst., 142, p. 103755. https://doi.org/10.1016/j.robot.2021.103755
  9. Fu, Q., Wang, J., Yu, H., Ali, I., Guo, F., He, Y. and Zhang, H. (2020), "PL-VINS: Real-Time Monocular Visual-Inertial SLAM with Point and Line Features", [Online]. Available: http://arxiv.org/abs/2009.07462
  10. Gordo, A., Almazan, J., Revaud, J. and Larlus, D. (2016), "Deep image retrieval: Learning global representations for image search", In: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14 (pp. 241-257). https://doi.org/10.1007/978-3-319-46466-4_15
  11. He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification", Proceedings of the IEEE international conference on computer vision, pp. 1026-1034.
  12. Huthwohl, P., Brilakis, I., Borrmann, A. and Sacks, R. (2018), "Integrating RC bridge defect information into BIM models", J. Comput. Civil Eng., 32(3), p. 04018013. https://doi.org/10.1061/(asce)cp.1943-5487.0000744
  13. Japan Road Bureau (MLIT) (2018), "Roads in Japan 2018", Minist. Land, Infastructure, Transp. Tour. Japan, pp. 2-39. [Online]. Available: http://www.mlit.go.jp/road/road_e/index_e.html
  14. Kingma, D.P. and Ba, J. (2015), "Adam: A method for stochastic optimization", Proceedings of the 3rd International Conference on Learning Represent. ICLR 2015 - Conf. Track Proc., pp. 1-15.
  15. Kwon, O.S., Park, C.S. and Lim, C.R. (2014), "A defect management system for reinforced concrete work utilizing BIM, image-matching and augmented reality", Autom. Constr., 46, 74-81. https://doi.org/10.1016/j.autcon.2014.05.005
  16. Li, S., Zhao, X. and Zhou, G. (2019), "Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network", Comput. Civil Infrastruct. Eng., 34(7), 616-634. https://doi.org/10.1111/mice.12433
  17. Lucas, B.D. and Kanade, T. (1981), "An iterative image registration technique with an application to stereo vision", Vol. 2, pp. 674-679.
  18. May, K.W., KC, C., Ochoa, J.J., Gu, N., Walsh, J., Smith, R.T. and Thomas, B.H. (2022), "The Identification, Development, and Evaluation of BIM-ARDM: A BIM-Based AR Defect Management System for Construction Inspections", Buildings, 12(2), p. 140.
  19. Qin, T., Li, P. and Shen, S. (2018), "Vins-mono: A robust and versatile monocular visual-inertial state estimator", IEEE Trans. Robot., 34(4), 1004-1020. https://doi.org/10.1109/TRO.2018.2853729
  20. Ranganathan, A. (2004), "The Levenberg-Marquardt Algorithm 3 LM as a blend of Gradient descent and Gauss-Newton itera", Internet httpexcelsior cs ucsb educoursescs290ipdfL MA pdf, Vol. 142, pp. 1-5. [Online]. Available: http://twiki.cis.rit.edu/twiki/pub/Main/AdvancedDipTeamB/thelevenberg-marquardt-algorithm.pdf
  21. Revaud, J., Weinzaepfel, P., De Souza, C., Pion, N., Csurka, G., Cabon, Y. and Humenberger, M. (2019), "R2D2: Repeatable and reliable detector and descriptor", Adv. Neural Inf. Process. Syst., vol. 32, no. NeurIPS.
  22. Ribeiro, D., Santos, R., Shibasaki, A., Montenegro, P., Carvalho, H. and Calcada, R. (2020), "Remote inspection of RC structures using unmanned aerial vehicles and heuristic image processing", Eng. Fail. Anal., 117, p. 1048130. https://doi.org/10.1016/j.engfailanal.2020.104813
  23. Schonberger, J.L. and Frahm, J.M. (2016), "Structure-from-motion revisited", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104-4113. https://doi.org/10.1109/CVPR.2016.445
  24. Tan, Y., Li, G., Cai, R., Ma, J. and Wang, M. (2022), "Mapping and modelling defect data from UAV captured images to BIM for building external wall inspection", Autom. Constr., 139, p. 104284. https://doi.org/10.1016/j.autcon.2022.104284
  25. Valinejadshoubi, M., Bagchi, A. and Moselhi, O. (2019), "Development of a BIM-based data management system for structural health monitoring with application to modular buildings: Case study", J. Comput. Civ. Eng., 33(3), p. 05019003. https://doi.org/10.1061/(asce)cp.1943-5487.0000826
  26. Von Gioi, R.G., Jakubowicz, J., Morel, J.M. and Randall, G. (2010), "LSD: A fast line segment detector with a false detection control", 32(4), 722-732. https://doi.org/10.1109/TPAMI.2008.300
  27. Yamane, T., Chun, P.J. and Honda, R. (2022), "Detecting and localising damage based on image recognition and structure from motion, and reflecting it in a 3D bridge model", Struct. Infrastruct. Eng., pp. 1-13. https://doi.org/10.1080/15732479.2022.2131845
  28. Zaccolo, M. (2002), "Good features to track", Methods Mol. Biol., 178, pp. 255-258. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6755905%5Cnhttp://www.ncbi.nlm.nih.gov/pubmed/11968495
  29. Zhang, L. and Koch, R. (2013), "An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency", J. Vis. Commun. Image Represent., 24(7), 794-805. https://doi.org/10.1016/j.jvcir.2013.05.006