Augmented Reality Framework for Data Visualization Based on Object Detection and Digital Twins

  • Pham, Hung (Department of Civil and Environmental Engineering, Myongji University) ;
  • Nguyen, Linh (Department of Civil and Environmental Engineering, Myongji University) ;
  • Huynh, Nhut (Department of Civil and Environmental Engineering, Myongji University) ;
  • Lee, Yong-Ju (Department of Civil and Environmental Engineering, Myongji University) ;
  • Park, Man-Woo (Department of Civil and Environmental Engineering, Myongji University)
  • Published : 2022.06.20

Abstract

While pursuing digitalization and paperless projects, the construction industry needs to settle on how to make the most of digitized data and information. On-site workers, who currently rely on paper documents to check and review design and construction plans, will need alternative ways to efficiently access the information without using any paper. Augmented Reality is a potential solution where the information customized to a user is aligned with the physical world. This paper proposes the Augmented Reality framework to deliver the information on on-site resources (e.g., workers and equipment) using head-mounted devices. The proposed framework was developed by interoperating Augmented Reality-supported devices and a digital twin platform in which all information related to ongoing tasks is accumulated in real-time. On-site resources appearing in the user's field of view are automatically detected by an object detection algorithm and then assigned to the corresponding information by matching the data in the digital twin platform. Preliminary experiments show the feasibility of the proposed framework. Worker detection results can be visualized on HoloLens 2 in near real-time, and the matching process obtained the accuracy greater than 88%.

Keywords

Acknowledgement

This work was conducted with the support of the National R&D Project for Smart Construction Technology (22SMIP-A158708-03) funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure and Transport, and managed by the Korean Expressway Corporation.