Journal of the Architectural Institute of Korea (대한건축학회논문집)

Architectural Institute of Korea (대한건축학회)
권호 표지
DOI QR코드

DOI QR Code

Proposal and Verification of the Faster R-CNN Regarding the Worker and Machine Interference Scope Detection Model to Prevent On-site Safety Accidents

현장 안전사고 예방을 위한 Faster R-CNN 기반 작업자와 기계 상호간섭 범위탐지 모델 제안 및 검증

  • Wang, Zepu (Dept. of Architectural Engineering, Hanyang University) ;
  • Kim, Jang-Soon (Architectural Engineering, Hanyang University) ;
  • Ham, Nam-Hyuk (Dept. of Digital Architectural and Urban Engineering, Hanyang Cyber University) ;
  • Kim, Jae-Jun (Dept. of Architectural Engineering, Hanyang University)
  • 왕택보 (한양대 건축공학과) ;
  • 김장순 (한양대 건축공학과) ;
  • 함남혁 (한양사이버대 디지털건축도시공학과) ;
  • 김재준 (한양대 건축공학과)
  • Received : 2021.12.01
  • Accepted : 2022.04.05
  • Published : 2022.04.30
⟨ Previous Next ⟩

Abstract

Safety management of construction projects have a significant impact on the construction project's schedule and the control carried out on site. Current site safety monitoring methods are highly dependent on manual labor; human errors can occur through missing content. This study aims to resolve these issues by applying machine learning visual detection algorithms to identify unsafe behaviors of workers at construction sites, to enhance external monitoring of workers and to relatively reduce the occurrence of safety accidents. A proposed method combines an object detection algorithm and spatial localization relationship definition. Only the machinery and workers at the construction site need to be accurately detected and the definition of spatial location relationship can be used to identify dangerous behaviors. A monitoring network framework suitable for this study was constructed with the environmental characteristics and image features of a construction site. The machines and workers were detected from construction images based on the Faster R-CNN algorithm for a computer to obtain the visual detection data from the construction site. Three spatial concepts were defined to determine the position relationships of machines and workers in these images. The detected location information of machines and workers at the construction site were combined and presented in a visualized form. Based on the results of this research, it confirmed that the method and performance were suitable for construction site safety management, which is expected to contribute to the speed, level of accuracy and risk warning with the application of automated progress monitoring methods.

Keywords

Acknowledgement

이 성과는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. 2021R1F1A1052050)

References

  1. Adillah, S. U., & Purnawan, A. (2020). Analysis of the Fulfillment of Labor Social Security as a Work of Legal Protection. Journal of Morality and Legal Culture, 1(1), 32-37. https://doi.org/10.20961/jmail.v1i1.44757
  2. Akinlolu, M., Haupt, T. C., Edwards, D. J., & Simpeh, F. (2020). A bibliometric review of the status and emerging research trends in construction safety management technologies. International Journal of Construction Management, 1-13.
  3. Bansal, V. K. (2011). Application of geographic information systems in construction safety planning. International Journal of Project Management, 29(1), 66-77. https://doi.org/10.1016/j.ijproman.2010.01.007
  4. Baker, H., Hallowell, M. R., & Tixier, A. J. P. (2020). AI-based prediction of independent construction safety outcomes from universal attributes. Automation in Construction, 118, 103146. https://doi.org/10.1016/j.autcon.2020.103146
  5. Cobo, L. C., Isbell Jr, C. L., & Thomaz, A. L. (2013). Object focused q-learning for autonomous agents. Georgia Institute of Technology.
  6. Chakkravarthy, R. (2019). Artificial intelligence for construction safety. Professional Safety, 64(1), 46.
  7. Carbonari, A., Giretti, A., & Naticchia, B. (2011). A proactive system for real-time safety management in construction sites. Automation in construction, 20(6), 686-698. https://doi.org/10.1016/j.autcon.2011.04.019
  8. Chian, E., Fang, W., Goh, Y. M., & Tian, J. (2021). Computer vision approaches for detecting missing barricades. Automation in Construction, 131, 103862. https://doi.org/10.1016/j.autcon.2021.103862
  9. Ding, L., Fang, W., Luo, H., Love, P. E., Zhong, B., & Ouyang, X. (2018). A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory. Automation in construction, 86, 118-124. https://doi.org/10.1016/j.autcon.2017.11.002
  10. Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., & An, W. (2018). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in Construction, 85, 1-9. https://doi.org/10.1016/j.autcon.2017.09.018
  11. Gheisari, M., & Esmaeili, B. (2016). Unmanned aerial systems (UAS) for construction safety applications. In Construction Research Congress 2016 (pp. 2642-2650).
  12. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Machine learning basics. Deep learning, 1(7), 98-164.
  13. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448).
  14. Gong, J., & Caldas, C. H. (2010). Computer vision-based video interpretation model for automated productivity analysis of construction operations. Journal of Computing in Civil Engineering, 24(3), 252-263. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000027
  15. Goertzel, B. (2007). Artificial general intelligence (Vol. 2). C. Pennachin (Ed.). New York: Springer.
  16. Gu, H. M., Seo, J. H., & Choo, S. Y. (2019). A Development of Facade Dataset Construction Technology Using Deep Learning-based Automatic Image Labeling. Journal of the Architectural Institute of Korea Planning & Design, 35(12), 43-53. https://doi.org/10.5659/JAIK_PD.2019.35.12.43
  17. Han, G., Oh, T. M., Kim, H., Song, K. I., Kim, Y., & Kwon, T. H. (2019). Determination of Crack Signals Using the Deep Learning Technique Based on a 1D Convolutional Neural Network for Smart Detection of Structural Damage Cracking. Journal of the Korean Society of Hazard Mitigation, 19(4), 1-7.
  18. Han, J., Zhang, D., Cheng, G., Liu, N., & Xu, D. (2018). Advanced deep-learning techniques for salient and category-specific object detection: a survey. IEEE Signal Processing Magazine, 35(1), 84-100. https://doi.org/10.1109/msp.2017.2749125
  19. Heinrich, H. W. (1980). Industrial Prevention: A Safety management Approach.
  20. Kang, K., & Ryu, H. (2019). Predicting types of occupational accidents at construction sites in Korea using random forest model. Safety Science, 120, 226-236. https://doi.org/10.1016/j.ssci.2019.06.034
  21. Klein, J. A. (2009). Two centuries of process safety at DuPont. Process Safety Progress, 28(2), 114-122. https://doi.org/10.1002/prs.10309
  22. Kymmell, W. (2008). Building information modeling: Planning and managing construction projects with 4D CAD and simulations (McGraw-Hill construction series). McGraw-Hill Education.
  23. Khosravi, Y., Asilian-Mahabadi, H., Hajizadeh, E., Hassanzadeh-Rangi, N., Bastani, H., & Behzadan, A. H. (2014). Factors influencing unsafe behaviors and accidents on construction sites: A review. International journal of occupational safety and ergonomics, 20(1), 111-125. https://doi.org/10.1080/10803548.2014.11077023
  24. Konstantinou, E., & Brilakis, I. (2018). Matching construction workers across views for automated 3D vision tracking on-site. Journal of Construction Engineering and Management, 144(7), 04018061. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001508
  25. Kolar, Z., Chen, H., & Luo, X. (2018). Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images. Automation in Construction, 89, 58-70. https://doi.org/10.1016/j.autcon.2018.01.003
  26. Lee, J. H., Lee, R. W., Hong, S. T., & Kim, Y. G. (2020). Image Processing System based on Deep Learning for Safety of Heat Treatment Equipment. The Journal of the Institute of Internet, Broadcasting and Communication, 20(6), 77-83. https://doi.org/10.7236/JIIBC.2020.20.6.77
  27. Lee, H. S., Lee, K. P., Park, M. S., Kim, H. S., & Lee, S. B. (2009). A Construction safety management system based on Building Information Modeling and Real-time Locating System. Korean Journal of Construction Engineering and Management, 10(6), 135-145.
  28. Li, R. Y. M., & Ng, D. P. L. (2017). Wearable robotics, industrial robots and construction worker's safety and health. In International Conference on Applied Human Factors and Ergonomics (pp. 31-36). Springer, Cham.
  29. Liu, H., & Tian, G. (2019). Building engineering safety risk assessment and early warning mechanism construction based on distributed machine learning algorithm. Safety Science, 120, 764-771. https://doi.org/10.1016/j.ssci.2019.08.022
  30. Lu, W., Huang, G. Q., & Li, H. (2011). Scenarios for applying RFID technology in construction project management. Automation in construction, 20(2), 101-106. https://doi.org/10.1016/j.autcon.2010.09.007
  31. Lee, J. M., Park, S. H., Cho, S. H., & Kim, J. H. (2021). Comparison of Models to Forecast Real Estates Index Introducing Machine Learning. Journal of the Architectural Institute of Korea, 37(1), 191-199. https://doi.org/10.5659/JAIK.2021.37.1.191
  32. Na, S., Xumin, L., & Yong, G. (2010). Research on k-means clustering algorithm: An improved k-means clustering algorithm. In 2010 Third International Symposium on intelligent information technology and security informatics (pp. 63-67). Ieee.
  33. O'Byrne, M., Schoefs, F., Ghosh, B., & Pakrashi, V. (2013). Texture analysis based damage detection of ageing infrastructural elements. Computer-Aided Civil and Infrastructure Engineering, 28(3), 162-177. https://doi.org/10.1111/j.1467-8667.2012.00790.x
  34. Poh, C. Q., Ubeynarayana, C. U., & Goh, Y. M. (2018). Safety leading indicators for construction sites: A machine learning approach. Automation in construction, 93, 375-386. https://doi.org/10.1016/j.autcon.2018.03.022
  35. Peng, X., & Schmid, C. (2016). Multi-region two-stream R-CNN for action detection. In European conference on computer vision (pp. 744-759). Springer, Cham.
  36. Perlman, A., Sacks, R., & Barak, R. (2014). Hazard recognition and risk perception in construction. Safety science, 64, 22-31. https://doi.org/10.1016/j.ssci.2013.11.019
  37. Park, C. S., & Kim, H. J. (2013). A framework for construction safety management and visualization system. Automation in Construction, 33, 95-103. https://doi.org/10.1016/j.autcon.2012.09.012
  38. Rho, J., Park, M., & Lee, H. S. (2020). Automated Construction Progress Management Using Computer Vision-based CNN Model and BIM. Korean Journal of Construction Engineering and Management, 21(5), 11-19. https://doi.org/10.6106/KJCEM.2020.21.5.011
  39. Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
  40. Roberts, D., Bretl, T., & Golparvar-Fard, M. (2017). Detecting and classifying cranes using camera-equipped UAVs for monitoring crane-related safety hazards. In Computing in Civil Engineering 2017 (pp. 442-449).
  41. Rubaiyat, A. H., Toma, T. T., Kalantari-Khandani, M., Rahman, S. A., Chen, L., Ye, Y., & Pan, C. S. (2016). Automatic detection of helmet uses for construction safety. In 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW) (pp. 135-142). IEEE.
  42. Shao, B., Hu, Z., Liu, Q., Chen, S., & He, W. (2019). Fatal accident patterns of building construction activities in China. Safety science, 111, 253-263. https://doi.org/10.1016/j.ssci.2018.07.019
  43. Shin, J., Kim, Y., & Kim, C. (2021). The perception of occupational safety and health (OSH) regulation and innovation efficiency in the construction industry: evidence from South Korea. International journal of environmental research and public health, 18(5), 2334. https://doi.org/10.3390/ijerph18052334
  44. Sakhakarmi, S., Park, J., & Cho, C. (2019). Enhanced machine learning classification accuracy for scaffolding safety using increased features. Journal of construction engineering and management, 145(2), pp. 04018133. https://doi.org/10.1061/(asce)co.1943-7862.0001601
  45. Seong, H., Son, H., & Kim, C. (2018). A comparative study of machine learning classification for color-based safety vest detection on construction-site images. KSCE Journal of Civil Engineering, 22(11), 4254-4262. https://doi.org/10.1007/s12205-017-1730-3
  46. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
  47. Tixier, A. J. P., Hallowell, M. R., Rajagopalan, B., & Bowman, D. (2016). Application of machine learning to construction injury prediction. Automation in construction, 69, 102-114. https://doi.org/10.1016/j.autcon.2016.05.016
  48. Tian, S. C., Li, H. X., & Wang, L. (2006). Three types hazard theory and prevention of coalmine accidents. Meitan Xuebao, 31(6), 706-710.
  49. Tang, S., & Golparvar-Fard, M. (2021). Machine Learning-Based Risk Analysis for Construction Worker Safety from Ubiquitous Site Photos and Videos. Journal of Computing in Civil Engineering, 35(6), pp. 04021020. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000979
  50. Uijlings, J. R., Van De Sande, K. E., Gevers, T., & Smeulders, A. W. (2013). Selective search for object recognition. International journal of computer vision, 104(2), 154-171. https://doi.org/10.1007/s11263-013-0620-5
  51. Wu, L., Mokhtari, S., Nazef, A., Nam, B., & Yun, H. B. (2016). Improvement of crack-detection accuracy using a novel crack defragmentation technique in image-based road assessment. Journal of Computing in Civil Engineering, 30(1), 04014118. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000451
  52. Xu, Y., Zhou, Y., Sekula, P., & Ding, L. (2021). Machine learning in construction: From shallow to deep learning. Developments in the Built Environment, 6, 100045. https://doi.org/10.1016/j.dibe.2021.100045
  53. Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833). Springer, Cham.
  54. Zhu, R., Hu, X., Hou, J., & Li, X. (2021). Application of machine learning techniques for predicting the consequences of construction accidents in China. Process Safety and Environmental Protection, 145, 293-302. https://doi.org/10.1016/j.psep.2020.08.006