Enhancing Automated Recognition of Small-Sized Construction Tools Using Synthetic Images: Validating Practical Applicability Through Confidence Scores

Computer vision techniques have been widely employed in automated construction management to enhance safety and prevent accidents at construction sites. However, previous research in the field of vision-based approaches has often overlooked small-sized construction tools. These tools present unique challenges in data collection due to their diverse shapes and sizes, as well as in improving model performance to accurately detect and classify them. To address these challenges, this study aimed to enhance the performance of vision-based classifiers for small-sized construction tools, including bucket, cord reel, hammer, and tacker, by leveraging synthetic images generated from a 3D virtual environment. Three classifiers were developed using the YOLOv8 algorithm, each differing in the composition of the training dataset: (i) 'Real-4000', trained on 4,000 authentic images collected through web crawling methods (1,000 images per object); (ii) 'Hybrid-4000', consisting of 2,000 authentic images and 2,000 synthetic images; and (iii) 'Hybrid-8000', incorporating 4,000 authentic images and 4,000 synthetic images. To validate the performance of the classifiers, 144 directly-captured images for each object were collected from real construction sites as the test dataset. The mean Average Precision at an IoU threshold of 0.5 (mAP_0.5) for the classifiers was 79.6%, 90.8%, and 94.8%, respectively, with the 'Hybrid-8000' model demonstrating the highest performance. Notably, for objects with significant shape variations, the use of synthetic images led to the enhanced performance of the vision-based classifiers. Moreover, the practical applicability of the proposed classifiers was validated through confidence scores, particularly between the 'Hybrid-4000' and 'Hybrid-8000' models. Statistical analysis using t-tests indicated that the performance of the 'Hybrid-4000' model would either matched or exceeded that of the 'Hybrid-8000'model based on confidence scores. Thus, employing the 'Hybrid-4000' model may be preferable in terms of data collection efficiency and processing time, contributing to enhanced safety and real-time automation and robotics in construction practices.