Enhancing Automated Multi-Object Tracking with Long-Term Occlusions across Consecutive Frames for Heavy Construction Equipment

Recent advances in artificial intelligence technology have led to active research aimed at systematically managing the productivity and environmental impact of major management targets such as heavy equipment at construction sites. However, challenges arise due to phenomena like partial occlusions, resulting from the dynamic working environment of construction sites (e.g., equipment overlapping, obstruction by structures), which impose practical constraints on precisely monitoring heavy equipment. To address these challenges, this study aims to enhance automated multi-object tracking (MOT) in scenarios involving long-term occlusions across consecutive frames for heavy construction equipment. To achieve this, two methodologies are employed to address long-term occlusions at construction sites: (i) tracking-by-detection and (ii) video inpainting with generative adversarial networks (GANs). Firstly, this study proposes integrating FairMOT with a tracking-by-detection algorithm like ByteTrack or SMILEtrack, demonstrating the robustness of re-identification (Re-ID) in occlusion scenarios. This method maintains previously assigned IDs when heavy equipment is temporarily obscured and then reappears, analyzing location, appearance, or motion characteristics across consecutive frames. Secondly, adopting video inpainting with GAN algorithms such as ProPainter is proposed, demonstrating robustness in removing objects other than the target object (e.g., excavator) during the video preprocessing and filling removed areas using information from surrounding pixels or other frames. This approach addresses long-term occlusion issues by focusing on a single object rather than multiple objects. Through these proposed approaches, improvements in the efficiency and accuracy of detection, tracking, and activity recognition for multiple heavy equipment are expected, mitigating MOT challenges caused by occlusions in dynamic construction site environments. Consequently, these approaches are anticipated to play a significant role in systematically managing heavy equipment productivity, environmental impact, and worker safety through the development of advanced construction and management systems.