KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Dynamic Manipulation of a Virtual Object in Marker-less AR system Based on Both Human Hands

  • Received : 2010.05.07
  • Accepted : 2010.06.27
  • Published : 2010.08.27
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a novel approach to control the augmented reality (AR) objects robustly in a marker-less AR system by fingertip tracking and hand pattern recognition. It is known that one of the promising ways to develop a marker-less AR system is using human's body such as hand or face for replacing traditional fiducial markers. This paper introduces a real-time method to manipulate the overlaid virtual objects dynamically in a marker-less AR system using both hands with a single camera. The left bare hand is considered as a virtual marker in the marker-less AR system and the right hand is used as a hand mouse. To build the marker-less system, we utilize a skin-color model for hand shape detection and curvature-based fingertip detection from an input video image. Using the detected fingertips the camera pose are estimated to overlay virtual objects on the hand coordinate system. In order to manipulate the virtual objects rendered on the marker-less AR system dynamically, a vision-based hand control interface, which exploits the fingertip tracking for the movement of the objects and pattern matching for the hand command initiation, is developed. From the experiments, we can prove that the proposed and developed system can control the objects dynamically in a convenient fashion.

Keywords

References

  1. M. Pressigout and E. Marchand, " Hybrid tracking algorithm for planar and non-planar structures subject to illumination change," in ISMAR'06, pp. 52-56, 2006.
  2. H. Kato and M. Billinghurst, "Marker tracking and HMD calibration for a video-based augmented reality conferencing system" in IWAR'99, pp. 85-94, 1999.
  3. D. Stricker, D. Klinker, and D. Reiners, "A fast and robust line-based optical tracker for augmented reality application," in IWAR'98, pp. 31-46, 1998.
  4. Y. Cho, J. Lee, and U. Neumann, "A multi-ring fiducial system and an intensity-invariant detection method for scalable augment reality," in IWAR'98, pp. 147-156, 1998.
  5. S. Vogt, A. Khamene, F. Sauer, and H. Niemann, "Single camera tracking of marker clusters: Multiparameter cluster optimization and experimental verification," in SIMAR'02, pp. 127-136, 2002.
  6. A. Comport, E. Marchand, and F. Chaumette, "A real-time tracker for markerless augment reality," in ISMAR'03, pp. 36-45, 2003.
  7. H. Kato, M. Billinghurst,I. Poupyrev, K. Imamoto, and K. Tachibana, "Virtual object manipulation on a table-top AR environments," in ISAR'00, pp. 111-119, 2000.
  8. M. Kolsch and M. Turk, "Fast 2D hand tracking with flocks of features and multi-cue integration," in CVPRW'04, pp. 158, 2004.
  9. T. Kurata, T. Okuma, M. Kourogi, and K. Sakaue, "The hand mouse: GMM hand-color classification and mean shift tracking," in 2nd Int'l Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time System, pp. 119-124, 2001.
  10. W. Lee and J. Park, "Augmented foam: A tangible augmented reality for product design," in ISMAR'05, pp. 106-109, 2005.
  11. J. Letessier and F. Berard, "Visual tracking of bare fingers for interactive surface," in Proc. ACM symposium on User interface software and technology, pp. 119-122, 2004.
  12. W. Mayol, A. Davison, B. Tordoff, N. Molton, and D. Murray, "Interaction between hand and wearable camera in 2D and 3D environments," in BMVA'04, 2004.
  13. J. Terrillon, M. Shirazi, H. Fukamachi, and S. Akamatsu, "Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images," in Proc. 9th Conference on Automatic Face and Gesture Recognition, pp. 54-61, 2000.
  14. T. Lee and T. Hollerer, "Hand AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking," in Proc. IEEE ISWC'07, pp. 83-90, 2007.
  15. T. Lee and T. Hollerer, "Initializing Markerless Tracking Using a Simple Hand Gesture," in Proc. ACM/IEEE ISMAR'07, pp. 1-2, 2007.
  16. T. Lee and T. Hollerer, "Hybrid Feature Tracking and User Interaction for Markerless Augment Reality," in IEEE VR'08 (10th Int'l Conference on Virtual Reality), pp. 145-152, 2008.
  17. BS. Lee and JC Chun "Manipulation of virtual objects in marker-less AR system by fingertip tracking and hand gesture recognition," in Int'l Conference on Interaction Science, pp. 1110-1115, 2009.
  18. H. Ishii and B. Ullmer, "Tangible bit: towards seamless interface between people, bits and atoms," in ACM CHI'97, pp. 234-241, 1977.
  19. K. Min and J. Chun, "A nonparametric skin color model for face detection from color images," in PDCAT'04, pp. 115-119, 2004.
  20. G. Borgefors, "Distance transformations in digital images," Computer Vision, Graphics and Image Processing, vol. 34, pp. 344-371, 1986. https://doi.org/10.1016/S0734-189X(86)80047-0
  21. M. Black, "Robust incremental optical flow," Ph.D thesis, Yale University, 1992.
  22. Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no.11, pp. 1330-1334, 2000. https://doi.org/10.1109/34.888718
  23. C.C Chen and H.T Chu, "Similarity Measurement Between Images," in Proc. of the 29th Annual International Computer Software and Applications Conference (COMPSAC'05), vol.1, pp. 41-41, 2005.
  24. F. J. Anscombe, "Graphs in Statistical Analysis," American Statistician, vol. 27, no. 1, pp. 17-21, 1973. https://doi.org/10.2307/2682899
  25. HJ. Park, DS. Kim, and JC Chun, "An Interactive Method for Volumetric Human Organ Model on Augmented Reality," in Proc. of APICT-IST and ICONI 2009, pp. 383-386, 2009.

Cited by

  1. 증강현실에서 OOK 기반의 CEM 맥락 데이터 생성 및 추출 기술 vol.12, pp.4, 2010, https://doi.org/10.5392/jkca.2012.12.04.020
  2. 비마커 증강현실을 위한 색상 및 깊이 정보를 융합한 Mean-Shift 추적 기반 손 자세의 추정 vol.17, pp.7, 2010, https://doi.org/10.9708/jksci.2012.17.7.155
  3. 증강현실 환경에서 손 가림 해결을 위한 피부 색상 정보 획득 vol.19, pp.2, 2010, https://doi.org/10.7315/cadcam.2014.111