International Journal of Advanced Culture Technology

The International Promotion Agency of Culture Technology (국제문화기술진흥원)
권호 표지
DOI QR코드

DOI QR Code

Augmented Reality Service Based on Object Pose Prediction Using PnP Algorithm

  • Kim, In-Seon (Department of Smart System, Kwangwoon University) ;
  • Jung, Tae-Won (Department of Immersive Content Convergence, Kwangwoon University) ;
  • Jung, Kye-Dong (Ingenium College of Liberal Arts, Kwangwoon University)
  • Received : 2021.10.26
  • Accepted : 2021.12.07
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Digital media technology is gradually developing with the development of convergence quaternary industrial technology and mobile devices. The combination of deep learning and augmented reality can provide more convenient and lively services through the interaction of 3D virtual images with the real world. We combine deep learning-based pose prediction with augmented reality technology. We predict the eight vertices of the bounding box of the object in the image. Using the predicted eight vertices(x,y), eight vertices(x,y,z) of 3D mesh, and the intrinsic parameter of the smartphone camera, we compute the external parameters of the camera through the PnP algorithm. We calculate the distance to the object and the degree of rotation of the object using the external parameter and apply to AR content. Our method provides services in a web environment, making it highly accessible to users and easy to maintain the system. As we provide augmented reality services using consumers' smartphone cameras, we can apply them to various business fields.

Keywords

Acknowledgement

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2020-0-00192, Development of Open Anchor-based AR Cloud for Universal Mixed Reality Service)

References

  1. Hyung-chul Kang, "A Study on Smart Tourism Content Using Augmented Reality," A Journal of Brand Design Association of Korea, Vol.17, No.3, pp. 165 - 174, Sep 2019. DOI: https://doi.org/10.18852/bdak.2 019.17.3.165
  2. Ki-Hwan Ko, "A Study on the Graphic Production Technology for AR Augmented Reality Game," The Journal of Korean Institute of Information Technology, Vol. 16, No. 11, pp. 123 - 132, Nov 2018. DOI: https://doi.org/10.5762/KAIS.2020.21.7.262
  3. Hyun-ju Oh, Ji-sol Jung, So-jung Park and Ki-Yong Lee, "Development of a tour information system for smart phones using CNN," Journal of the Korean Institute of Information Scientists and Engineers 2019 Korea Computer Science Competition, pp. 1642-1644, Jun 2019.
  4. Jong-in Choe, Jung-hyun Lee, Dong-wan Kang and Sang-hyun Seo, "AR based Beverage Information Visualization and Sharing System using Deep Learning," Journal of Digital Contents Society, Vol. 21, No. 3, pp. 445-452, Mar 2020. DOI: https://dx.doi.org/10.9728/dcs.2020.21.3.445
  5. Jin-Seon Oh and In-Gook Chun, "Implementation of Smart Shopping Cart using Object Detection Method based on Deep Learning," Journal of the Korea Academia-Industrial cooperation Society, Vol. 21, No. 7, pp. 262-269 July 2020. DOI: https://doi.org/10.5762/KAIS.2020.21.7.262
  6. Jin-Kyu Kang, "OpenXR and WebXR in Virtual Augmented Reality," Broadcasting and Media Magazine, Vol. 26, No. 1, pp. 12-18, Jan 2021.
  7. Chi-Seo Jeong, Jun-Sik Kim, Dong-Kyun Kim, Soon-Chul Kwon and Kye-Dong Jung, "AR Anchor System Using Mobile Based 3D GNN Detection," International Journal of Internet, Broadcasting and Communication, Vol.13, No.1, pp. 54-60, Feb 2021. DOI: http://dx.doi.org/10.7236/IJIBC.2021.13.1.54
  8. Bugra Tekin, Sudipta N. Sinha and Pascal Fua, "Real-Time Seamless Single Shot 6D Object Pose Prediction," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 292-301, Dec 2018.
  9. Vincent Lepetit, Francesc Moreno-Noguer and Pascal Fua, "EPnP: An Accurate O(n) Solution to the PnP problem," International Journal of Computer Vision, Vo. 81, No. 2, Article number. 155, Feb 2009. DOI: http://dx.doi.org/10.1007/s11263-008-0152-6
  10. Blair Maclntyre and Trevor F. Smith, "Thoughts on the Future of WebXR and the Immersive Web," 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 338-342, Apr 2019. DOI: https://doi.org/10.1109/ISMAR-Adjunct.2018.00099
  11. Dae-Hyeon Lee, "A Study on the Performance Evaluation of WebXR Device API in Augmented Reality Environment of Mobile Web," Master Thesis, Graduate School of Smart Convergence, Kwangwoon University, Korea, 2020.