• Journal of the Society of Disaster Information
• /
• v.20 no.2
• /
• pp.302-314
• /
• 2024

Purpose: This study proposes a fall detection model based on a top-down deep learning pose estimation model to automatically determine falls of multiple workers in an underground utility tunnel, and evaluates the performance of the proposed model. Method: A model is presented that combines fall discrimination rules with the results inferred from YOLOv8-pose, one of the top-down pose estimation models, and metrics of the model are evaluated for images of standing and falling two or fewer workers in the tunnel. The same process is also conducted for a bottom-up type of pose estimation model (OpenPose). In addition, due to dependency of the falling interference of the models on worker detection by YOLOv8-pose and OpenPose, metrics of the models for fall was not only investigated, but also for person. Result: For worker detection, both YOLOv8-pose and OpenPose models have F1-score of 0.88 and 0.71, respectively. However, for fall detection, the metrics were deteriorated to 0.71 and 0.23. The results of the OpenPose based model were due to partially detected worker body, and detected workers but fail to part them correctly. Conclusion: Use of top-down type of pose estimation models would be more effective way to detect fall of workers in the underground utility tunnel, with respect to joint recognition and partition between workers.

• The Journal of the Convergence on Culture Technology
• /
• v.8 no.5
• /
• pp.735-740
• /
• 2022

The construction is the industry with the highest fatalities, and the fatalities has not decreased despite various institutional improvements. Accordingly, real-time safety management by applying artificial intelligence (AI) to CCTV images is emerging. Although some research on worker detection by applying AI to images of construction sites is being conducted, there are limitations in performance expression due to problems such as complex background due to the nature of the construction industry. In this study, the YOLO model and the OpenPose model were fused to improve the performance of worker detection and posture estimation to improve the detection performance of workers in various complex conditions. This is expected to be highly useful in terms of unsafe behavior and health management of workers in the future.

• Journal of the Society of Disaster Information
• /
• v.19 no.3
• /
• pp.498-509
• /
• 2023

Purpose: This paper proposes a method detecting the falling of a maintenance worker in the underground utility tunnel, by applying deep learning techniques using CCTV video, and evaluates the applicability of the proposed method to the worker monitoring of the utility tunnel. Method: Each rule was designed to detect the falling of a maintenance worker by using the inference results from pre-trained YOLOv5 and OpenPose models, respectively. The rules were then integrally applied to detect worker falls within the utility tunnel. Result: Although the worker presence and falling were detected by the proposed model, the inference results were dependent on both the distance between the worker and CCTV and the falling direction of the worker. Additionally, the falling detection system using YOLOv5 shows superior performance, due to its lower dependence on distance and fall direction, compared to the OpenPose-based. Consequently, results from the fall detection using the integrated dual deep learning model were dependent on the YOLOv5 detection performance. Conclusion: The proposed hybrid model shows detecting an abnormal worker in the utility tunnel but the improvement of the model was meaningless compared to the single model based YOLOv5 due to severe differences in detection performance between each deep learning model

• IEMEK Journal of Embedded Systems and Applications
• /
• v.18 no.6
• /
• pp.303-310
• /
• 2023

A single vision system limits the ability to accurately understand the spatial constraints and interactions between robots and dynamic workers caused by gantry robots and collaborative robots during production manufacturing. In this paper, we propose a 3D pose registration method for dynamic workers based on a multi-domain vision system for safety monitoring in manufacturing environments. This method uses OpenPose, a deep learning-based posture estimation model, to estimate the worker's dynamic two-dimensional posture in real-time and reconstruct it into three-dimensional coordinates. The 3D coordinates of the reconstructed multi-domain vision system were aligned using the ICP algorithm and then registered to a single 3D coordinate system. The proposed method showed effective performance in a manufacturing process environment with an average registration error of 0.0664 m and an average frame rate of 14.597 per second.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.328-335
• /
• 2022

With the advance of robot capabilities and functionalities, construction robots assisting construction workers have been increasingly deployed on construction sites to improve safety, efficiency and productivity. For close-proximity human-robot collaboration in construction sites, robots need to be aware of the context, especially construction worker's behavior, in real-time to avoid collision with workers. To recognize human behavior, most previous studies obtained 3D human poses using a single camera or an RGB-depth (RGB-D) camera. However, single-camera detection has limitations such as occlusions, detection failure, and sensor malfunction, and an RGB-D camera may suffer from interference from lighting conditions and surface material. To address these issues, this study proposes a novel method of 3D human pose estimation by extracting 2D location of each joint from multiple images captured at the same time from different viewpoints, fusing each joint's 2D locations, and estimating the 3D joint location. For higher accuracy, the probabilistic representation is used to extract the 2D location of the joints, considering each joint location extracted from images as a noisy partial observation. Then, this study estimates the 3D human pose by fusing the probabilistic 2D joint locations to maximize the likelihood. The proposed method was evaluated in both simulation and laboratory settings, and the results demonstrated the accuracy of estimation and the feasibility in practice. This study contributes to ensuring human safety in close-proximity human-robot collaboration by providing a novel method of 3D human pose estimation.

• International conference on construction engineering and project management
• /
• 2024.07a
• /
• pp.775-782
• /
• 2024

This systematic review comprehensively analyzes the application of computer vision in construction productivity measurement and emphasizes the importance of worker accountability in construction sites. It identifies a significant gap in the connection level between input (resources) and output data (products or progress) of productivity monitoring, a factor not adequately addressed in prior research. The review highlights three fundamental groups: input, output, and connection groups. Object detection, tracking, pose, and activity recognition, as the input stage, are essential for identifying characteristics and worker movements. The output phase will mostly focus on progress monitoring, and understanding the interaction of workers with other entities will be discussed in the connection groups. This study offers four research future research directions for the worker accountability monitoring process, such as human-object interaction (HOI), generative AI, location-based management systems (LBMS), and robotic technologies. The successful accountability monitoring will secure the accuracy of productivity measurement and elevate the competitiveness of the construction industry.

• Journal of Korea Society of Industrial Information Systems
• /
• v.29 no.5
• /
• pp.21-32
• /
• 2024

This paper proposes a smart safety system that combines low-cost CW(Continuous Wave) radar and IMU sensors to enhance blind spots that pose safety risks to workers in industrial manufacturing environments. The system employs a 24 GHz radar and a 6-axis IMU sensor to detect worker movements and utilizes a machine learning model to recognize worker situations in vibrating manufacturing sites. The ensemble boosting tree-based model achieved over 92.8% worker detection accuracy, demonstrating its effectiveness in improving safety in industrial settings.

• Journal of the Korea Society of Computer and Information
• /
• v.29 no.9
• /
• pp.109-114
• /
• 2024

In this paper, we design and implement a worker motion 3D visualization module based on human sensors. The three key modules that make up this system are Human Sensor Implementation, Data Set Creation, and Visualization. Human Sensor Implementation provides the functions of setting and installing the human sensor locations and collecting worker motion data through the human sensors. Data Set Creation offers functions for converting and storing motion data, creating near real-time worker motion data sets, and processing and managing sensor and motion data sets. Visualization provides functions for visualizing the worker's 3D model, evaluating motions, calculating loads, and managing large-scale data. In worker 3D model visualization, motion data sets (Skeleton & Position) are synchronized and mapped to the worker's 3D model, and the worker's 3D model motion animation is visualized by combining the worker's 3D model with analysis results. The human sensor-based worker motion 3D visualization module designed and implemented in this paper can be widely utilized as a foundational technology in the smart factory field in the future.

• Journal of Korean Society of Occupational and Environmental Hygiene
• /
• v.34 no.2
• /
• pp.134-138
• /
• 2024

The rapid advancement of technologies within the Fourth Industrial Revolution is expected to bring significant changes to the work environment across industries. While automation and digitalization not only enhance productivity but also health and safety, they also pose new health risks such as isolation, technostress, and musculoskeletal disorders. The rise of remote work and platform labor necessitates strengthening social protections for workers' health rights. To ensure that technological progress leads to better worker health, proactive and comprehensive occupational health policies are essential. Cultivating interdisciplinary expertise in occupational health professionals is also crucial. Industrial hygienists can play a pivotal role in assessing and managing the health impacts of technological changes, as well as bridging the gap between innovation and worker well-being.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.22 no.1
• /
• pp.24-30
• /
• 2021

In this study, an augmented reality (AR) app that detects the screw-assembly state of a car valve-body and assists the assembly work is developed and the effectiveness of the app is shown through testing. The app creates the contents indicating the screw-assembly position and order, and the screw-assembly state. Then, the contents are registrated onto the valve-body image on a smart-phone screen to be shown to the worker during assembly. To this end, the features are extracted from the 2D image of the valve-body and the location of the valve-body is tracked. By extracting the areas where the screws are to be assembled, and periodically determining the luminance of these areas, it is checked whether the screws are assembled in order at the predetermined position of the valve-body. When an error is detected during assembly, a warning sound is notified to the worker, and the worker can check the assembly state on the smart-phone screen and handle the error, immediately. Study results found that it takes about 65 ms to detect the assembly state of the five screws, and the assembly state is detected without error for 1 hour.