International conference on construction engineering and project management (국제학술발표논문집)

  • 2024.07a
  • /
  • Pages.775-782
  • /
  • 2024
  • /
  • 2508-9048(eISSN)
Korea Institute of Construction Engineering and Management (한국건설관리학회)
권호 표지
DOI QR코드

DOI QR Code

Worker Accountability in Computer Vision for Construction Productivity Measurement: A Systematic Review

  • Mik Wanul KHOSIIN (Department of Civil Engineering, College of Engineering, National Taiwan University) ;
  • Jacob J. LIN (Department of Civil Engineering, College of Engineering, National Taiwan University) ;
  • Chuin-Shan CHEN (Department of Civil Engineering, College of Engineering, National Taiwan University)
  • Published : 2024.07.29
Download PDF
⟨ Previous Next ⟩

Abstract

This systematic review comprehensively analyzes the application of computer vision in construction productivity measurement and emphasizes the importance of worker accountability in construction sites. It identifies a significant gap in the connection level between input (resources) and output data (products or progress) of productivity monitoring, a factor not adequately addressed in prior research. The review highlights three fundamental groups: input, output, and connection groups. Object detection, tracking, pose, and activity recognition, as the input stage, are essential for identifying characteristics and worker movements. The output phase will mostly focus on progress monitoring, and understanding the interaction of workers with other entities will be discussed in the connection groups. This study offers four research future research directions for the worker accountability monitoring process, such as human-object interaction (HOI), generative AI, location-based management systems (LBMS), and robotic technologies. The successful accountability monitoring will secure the accuracy of productivity measurement and elevate the competitiveness of the construction industry.

Keywords

Acknowledgement

This research was supported by the Ministry of Education Taiwan (Grant number: MOE-TSP-110.08.16) and facilitated by the National Center for Research on Earthquake Engineering and the Department of Civil Engineering, National Taiwan University (NCREE-NTUCE) Joint Artificial Intelligence Research Center. This research is also funded by The Ministry of Science and Technology (MOST) through a project (Grant number: 108-WFA-0110-731).

References

  1. Project Management Institute, A guide to the project management body of knowledge (PMBOK® guide), vol. 40. 2008.
  2. P. Hee-Sung, T. S. R., and T. R. L., "Benchmarking of Construction Productivity," J Constr Eng Manag, vol. 131, no. 7, pp. 772-778, Jul. 2005, doi: 10.1061/(ASCE)0733-9364(2005)131:7(772).
  3. V. Gaspersz, "Manajemen Produktivitas Total Strategi Oeningkatan Produktivitas Bisnis Global," Gramedia Pustaka Utama, Jakarta, 1998.
  4. R. Farmer, "McKinsey & company," in Management Consultancy: What Next?, 2016. doi: 10.1057/9781403907189_24.
  5. A. Pal and S. H. Hsieh, "Deep-learning-based visual data analytics for smart construction management," Automation in Construction, vol. 131. Elsevier, p. 103892, 2021. doi: 10.1016/j.autcon.2021.103892.
  6. A. S. da Barbosa and D. B. Costa, "PRODUCTIVITY MONITORING OF CONSTRUCTION ACTIVITIES USING DIGITAL TECHNOLOGIES: A LITERATURE REVIEW," in IGLC 2021 - 29th Annual Conference of the International Group for Lean Construction - Lean Construction in Crisis Times: Responding to the Post-Pandemic AEC Industry Challenges, 2021. doi: 10.24928/2021/0141.
  7. W. S. Alaloul, K. M. Alzubi, A. B. Malkawi, M. Al Salaheen, and M. A. Musarat, "Productivity monitoring in building construction projects: a systematic review," Engineering, Construction and Architectural Management, vol. 29, no. 7, pp. 2760-2785, Jan. 2022, doi: 10.1108/ECAM-03-2021-0211.
  8. B. Sherafat et al., "Automated Methods for Activity Recognition of Construction Workers and Equipment: State-of-the-Art Review," J Constr Eng Manag, vol. 146, no. 6, 2020, doi: 10.1061/(asce)co.1943-7862.0001843.
  9. S. Archchana and W. Pan, "BUILDING INFORMATION MODELLING FOR CONSTRUCTION PRODUCTIVITY MEASUREMENT," in World Construction Symposium, 2023. doi: 10.31705/WCS.2023.25.
  10. A. S. M. Moohialdin, F. Lamari, M. Miska, and B. Trigunarsyah, "Construction worker productivity in hot and humid weather conditions: A review of measurement methods at task, crew and project levels," Engineering, Construction and Architectural Management, vol. 27, no. 1. 2020. doi: 10.1108/ECAM-05-2018-0191.
  11. K. Mhmoud Alzubi, W. Salah Alaloul, A. B. Malkawi, M. Al Salaheen, A. Hannan Qureshi, and M. Ali Musarat, "Automated monitoring technologies and construction productivity enhancement: Building projects case," Ain Shams Engineering Journal, vol. 14, no. 8, 2023, doi: 10.1016/j.asej.2022.102042.
  12. A. Rathnayake and C. Middleton, "Systematic Review of the Literature on Construction Productivity," J Constr Eng Manag, vol. 149, no. 6, 2023, doi: 10.1061/jcemd4.coeng-13045.
  13. T. H. Randolph, "Labor Productivity and Work Sampling: The Bottom Line," J Constr Eng Manag, vol. 117, no. 3, pp. 423-444, Sep. 1991, doi: 10.1061/(ASCE)0733-9364(1991)117:3(423).
  14. Z. Qilin, W. Zhichen, Y. Bin, L. Ke, Z. Binghan, and L. Boda, "Reidentification-Based Automated Matching for 3D Localization of Workers in Construction Sites," Journal of Computing in Civil Engineering, vol. 35, no. 6, p. 04021019, Nov. 2021, doi: 10.1061/(ASCE)CP.1943-5487.0000975.
  15. M. B. Eddine, A. Mohamad, and K. Hiam, "Vision-Based Framework for Intelligent Monitoring of Hardhat Wearing on Construction Sites," Journal of Computing in Civil Engineering, vol. 33, no. 2, p. 4018066, Mar. 2019, doi: 10.1061/(ASCE)CP.1943-5487.0000813.
  16. N. Wojke, A. Bewley, and D. Paulus, "Simple online and realtime tracking with a deep association metric," in Proceedings - International Conference on Image Processing, ICIP, 2017. doi: 10.1109/ICIP.2017.8296962.
  17. E. Konstantinou, J. Lasenby, and I. Brilakis, "Adaptive computer vision-based 2D tracking of workers in complex environments," Autom Constr, vol. 103, pp. 168-184, 2019, doi: https://doi.org/10.1016/j.autcon.2019.01.018.
  18. T. Ghazaleh, H. Amin, and B. Nizar, "Two-Dimensional and Three-Dimensional CNN-Based Simultaneous Detection and Activity Classification of Construction Workers," Journal of Computing in Civil Engineering, vol. 36, no. 4, p. 4022009, Jul. 2022, doi: 10.1061/(ASCE)CP.1943-5487.0001024.
  19. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016. doi: 10.1109/CVPR.2016.91.
  20. S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," IEEE Trans Pattern Anal Mach Intell, vol. 39, no. 6, 2017, doi: 10.1109/TPAMI.2016.2577031.
  21. K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask R-CNN," IEEE Trans Pattern Anal Mach Intell, vol. 42, no. 2, 2020, doi: 10.1109/TPAMI.2018.2844175.
  22. O. Kopuklu, X. Wei, and G. Rigoll, "You Only Watch Once: A Unified CNN Architecture for Real-Time Spatiotemporal Action Localization," CoRR, vol. abs/1911.06644, 2019, [Online]. Available: http://arxiv.org/abs/1911.06644
  23. M.-Y. Cheng, A. F. K. Khitam, and H. H. Tanto, "Construction worker productivity evaluation using action recognition for foreign labor training and education: A case study of Taiwan," Autom Constr, vol. 150, p. 104809, 2023, doi: https://doi.org/10.1016/j.autcon.2023.104809.
  24. J. J. Lin and M. Golparvar-Fard, "Construction Progress Monitoring Using Cyber-Physical Systems," in Cyber-Physical Systems in the Built Environment, 2020, pp. 63-87. doi: 10.1007/978-3-030-41560-0_5.
  25. G. A.S. and J. B. Edayadiyil, "Automated progress monitoring of construction projects using Machine learning and image processing approach," Mater Today Proc, vol. 65, pp. 554-563, 2022, doi: https://doi.org/10.1016/j.matpr.2022.03.137.
  26. L. J. J., H. K. K., and G.-F. Mani, "A Framework for Model-Driven Acquisition and Analytics of Visual Data Using UAVs for Automated Construction Progress Monitoring," Computing in Civil Engineering 2015. in Proceedings. pp. 156-164, Nov. 14, 2022. doi: doi:10.1061/9780784479247.020.
  27. L. J. J., H. K. K., and G.-F. Mani, "A Framework for Model-Driven Acquisition and Analytics of Visual Data Using UAVs for Automated Construction Progress Monitoring," Computing in Civil Engineering 2015. in Proceedings. pp. 156-164, Nov. 14, 2022. doi: doi:10.1061/9780784479247.020.
  28. N. Wang et al., "Exploring Spatio-Temporal Graph Convolution for Video-based Human-Object Interaction Recognition," IEEE Transactions on Circuits and Systems for Video Technology, p. 1, 2023, doi: 10.1109/TCSVT.2023.3259430.
  29. D. Purwanto, Y. T. Chen, and W. H. Fang, "First-Person Action Recognition with Temporal Pooling and Hilbert-Huang Transform," IEEE Trans Multimedia, vol. 21, no. 12, 2019, doi: 10.1109/TMM.2019.2919434.
  30. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "The graph neural network model," IEEE Trans Neural Netw, vol. 20, no. 1, 2009, doi: 10.1109/TNN.2008.2005605.
  31. P. Velickovic, A. Casanova, P. Lio, G. Cucurull, A. Romero, and Y. Bengio, "Graph attention networks," in 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, 2018. doi: 10.1007/978-3-031-01587-8_7.
  32. M. Tomei, L. Baraldi, S. Calderara, S. Bronzin, and R. Cucchiara, "Video action detection by learning graph-based spatio-temporal interactions," Computer Vision and Image Understanding, vol. 206, 2021, doi: 10.1016/j.cviu.2021.103187.
  33. Y. Huang, H. Bi, Z. Li, T. Mao, and Z. Wang, "STGAT: Modeling spatial-temporal interactions for human trajectory prediction," in Proceedings of the IEEE International Conference on Computer Vision, 2019. doi: 10.1109/ICCV.2019.00637.
  34. H. Tang, P. Wei, J. Li, and N. Zheng, "EvoSTGAT: Evolving spatiotemporal graph attention networks for pedestrian trajectory prediction," Neurocomputing, vol. 491, 2022, doi: 10.1016/j.neucom.2022.03.051.
  35. A. Vaswani et al., "Attention is all you need," in Advances in Neural Information Processing Systems, 2017.
  36. J. Park, J. W. Park, and J. S. Lee, "ViPLO: Vision Transformer Based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detection," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2023. doi: 10.1109/CVPR52729.2023.01645.
  37. Z. Liang, Y. Guan, and J. Rojas, "Visual-Semantic Graph Attention Network for Human-Object Interaction Detection," CoRR, vol. abs/2001.02302, 2020, [Online]. Available: http://arxiv.org/abs/2001.02302
  38. Z. Liang, J. Liu, Y. Guan, and J. Rojas, "Pose-based Modular Network for Human-Object Interaction Detection," CoRR, vol. abs/2008.02042, 2020, [Online]. Available: https://arxiv.org/abs/2008.02042
  39. R. Dominic, T. C. Wilfredo, T. Shuai, and G.-F. Mani, "Vision-Based Construction Worker Activity Analysis Informed by Body Posture," Journal of Computing in Civil Engineering, vol. 34, no. 4, p. 4020017, Jul. 2020, doi: 10.1061/(ASCE)CP.1943-5487.0000898.
  40. H. Yan, Y. Liu, L. Jin, and X. Bai, "The development,application,and future of LLM similar to ChatGPT," Journal of Image and Graphics, vol. 28, no. 9, 2023, doi: 10.11834/jig.230536.
  41. O. Seppanen, G. Ballard, and S. Pesonen, "The Combination of Last Planner System and Location-Based Management System," Lean Construction Journal, vol. 6, Jan. 2010.
  42. J. Ratajczak, M. Riedl, and D. T. Matt, "BIM-based and AR application combined with location-based management system for the improvement of the construction performance," Buildings, vol. 9, no. 5, 2019, doi: 10.3390/buildings9050118.
  43. R. Kenley and O. Seppanen, Location-Based Management for Construction: Planning, Scheduling and Control. 2016. doi: 10.4324/9780203030417.
  44. H. Huang, "Location Based Services," in Springer Handbooks, 2022. doi: 10.1007/978-3-030-53125-6_22.
  45. C.-J. Liang, X. Wang, V. R. Kamat, and C. C. Menassa, "Human-Robot Collaboration in Construction: Classification and Research Trends," J Constr Eng Manag, vol. 147, no. 10, 2021, doi: 10.1061/(asce)co.1943-7862.0002154.
  46. Y. Liu, M. Habibnezhad, and H. Jebelli, "Brainwave-driven human-robot collaboration in construction," Autom Constr, vol. 124, 2021, doi: 10.1016/j.autcon.2021.103556.
  47. M. Zhang, R. Xu, H. Wu, J. Pan, and X. Luo, "Human-robot collaboration for on-site construction," Automation in Construction, vol. 150. 2023. doi: 10.1016/j.autcon.2023.104812.