Journal of Internet Computing and Services (인터넷정보학회논문지)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지

Vision-based Motion Control for the Immersive Interaction with a Mobile Augmented Reality Object

모바일 증강현실 물체와 몰입형 상호작용을 위한 비전기반 동작제어

  • Received : 2011.03.01
  • Accepted : 2011.04.05
  • Published : 2011.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Vision-based Human computer interaction is an emerging field of science and industry to provide natural way to communicate with human and computer. Especially, recent increasing demands for mobile augmented reality require the development of efficient interactive technologies between the augmented virtual object and users. This paper presents a novel approach to construct marker-less mobile augmented reality object and control the object. Replacing a traditional market, the human hand interface is used for marker-less mobile augmented reality system. In order to implement the marker-less mobile augmented system in the limited resources of mobile device compared with the desktop environments, we proposed a method to extract an optimal hand region which plays a role of the marker and augment object in a realtime fashion by using the camera attached on mobile device. The optimal hand region detection can be composed of detecting hand region with YCbCr skin color model and extracting the optimal rectangle region with Rotating Calipers Algorithm. The extracted optimal rectangle region takes a role of traditional marker. The proposed method resolved the problem of missing the track of fingertips when the hand is rotated or occluded in the hand marker system. From the experiment, we can prove that the proposed framework can effectively construct and control the augmented virtual object in the mobile environments.

비전기반 인간컴퓨터 상호작용은 컴퓨터와 인간의 상호소통을 자연스럽게 제공하는 측면에서 과학과 산업분야에서 주목받는 연구 분야이다. 특히 최근 모바일 증강현실의 활용에 대한 수요증대는 증강된 가상 객체와 사용자간의 효과적인 상호작용 기술 개발을 요구하고 있다. 본 논문에서는 안드로이드기반 모바일 플랫폼에서 비 마커기반의 새로운 모바일 증강현실 객체생성 및 제어 방법을 제시하였다. 전통적인 마커를 대신하여 비 마커기반 모바일 증강현실 시스템에서는 사용자에게 친숙한 손 인터페이스를 이용하였다. 기존의 데스크 탑 기반의 증강현실에 비하여 제한된 시스템 사양을 이용하는 비 마커기반 모바일 증강현실을 구현하기 위하여 마커의 역할을 수행할 손의 최적영역을 모바일 장치의 카메라로부터 실시간으로 검출 하여 객체를 증강시키는 방법을 제시하였다. 손의 최적영역의 추출은 YCbCr 스킨컬러 모델을 이용한 손영역 검출과 Rotating Calipers 알고리즘을 적용한 최적 사각형 추출에 의해 구현하였다. 이때 추출된 최적 사각형은 통상적인 마커기반 증강현실에서의 마커역할을 수행할 수 있도록 하였으며, 기존의 손끝추적에 의한 마커 생성 시 발생하는 손의 회전이나 가려짐에 의한 문제를 해결하였다. 실험을 통하여 제안된 방법이 비 마커기반 모바일 객체를 효과적으로 생성 제어할 수 있음을 입증하였다.

Keywords

References

  1. J. Carmigniani, B. Furht, M. Anisetti, P. Ceravolo, E. Damiani and M. Ivkovic, "Augmented reality, technologies, systems, and applications", Multimedia Tools and Applications, Vol 51, Issue 1, pp. 341-377, 2011. https://doi.org/10.1007/s11042-010-0660-6
  2. D.W.F. van Krevelen and R. Poelman, "A survey of Augmented Reality Technologies, Applications, and Limitations", The Intl. Journal of Virtual Reality, Vol 9. No. 2, pp. 1-20, 2010.
  3. M. Fiala, "ARTag, a fiducial marker system using digital techniques", In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol 2, pp. 590-596, 2005.
  4. H. Kato and M. Billinghurst, "Marker tracking and HMD calibration for a video-based augmented reality conferencing system", In Proceedings on 2nd IEEE International Workshop on Augmented Reality, pp. 85-94, 1999.
  5. D. Wagner and D. Schmalstieg, "ARToolKitPlus for Pose Tracking on Mobile Devices", In Proc. of 12th Computer Vision Winter Workshop, pp. 139-146, 2007.
  6. D. Stricker, D. Klinker and D. Reiners, "A fast and robust line-based optical tracker for augmented reality application", In Proceedings on 1st IEEE International Workshop on Augmented Reality, pp. 31-46, 1998.
  7. Y. Cho, J. Lee and U. Neumann, "A multi-ring fiducial system and an intensityinvariant detection method for scalable augment reality", In Proceedings on 1st IEEE International Workshop on Augmented Reality, pp.147-156, 1998.
  8. S. Vogt, A. Khamene, F. Sauer and H. Niemann, "Single camera tracking of marker clusters: Multiparameter cluster optimization and experimental verification", In Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 127-136, 2002.
  9. Lee, T. and Hollerer, T., "Hand AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking", In Proceedings of IEEE International Symposium on Wearable Computers, pp. 83-90, 2007.
  10. Lee, T. and Hollerer, T., "Initializing Markerless Tracking Using a Simple Hand Gesture", In Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1-2, 2007.
  11. Lee, T. and Hollerer, T.,"Hybrid Feature Tracking and User Interaction for Markerless Augment Reality", In IEEE Int'l Conference on Virtual Reality, pp. 145-152, 2008.
  12. Lee, B and Chun, J, "Interactive manipulation of augmented objects in marker-less AR using vision-based hand mouse", In Int'l Conference on Information Technology, pp. 398-403, 2010.
  13. Chun, J and Lee, B, "Dynamic Manipulation of a Virtual Object in Marker-less AR system Based on Both Human Hands", TIIS Vol. 4, No.4, pp. 618-632, 2010.
  14. G. Klein and D. Murray, "Parallel tracking and mapping for small AR workspaces", In Proceedings of 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225-234, 2007.
  15. G. Klein and D. Murray, "Parallel Tracking and Mapping on a Camera Phone" In Proceedings of IEEE International Symposium on Mixed and Augmented Reality, pp. 83-86, 2009.
  16. D. Wagner, G. Reitmayr, A. Mulloni, Tom Drummond4, Dieter Schmalstieg, "Pose Tracking from Natural Features on Mobile Phones", In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 125 - 134, 2008.
  17. Min, K, Chun, J and Park, G, "A Nonparametric Skin Color Model for Face Detection from Color Images", LNCS Vol. 3320, pp. 115-119, 2004.
  18. G. Borgefors, "Distance transformations in digital images," Computer Vision, Graphics and Image Processing, Vol. 34, pp. 344-371, 1986 https://doi.org/10.1016/S0734-189X(86)80047-0
  19. Shamos, M.I. "Computational geometry", Yale University, PhD. thesis, 1978.
  20. Z. Zhang, "A flexible new technique for camera calibration", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 22, No.11, pp. 1330-1334, 2000. https://doi.org/10.1109/34.888718