Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Comparative Experiment of Hand-based Interface in Immersive Virtua Reality

몰입형 가상현실에서 손 기반 인터페이스의 비교 실험에 관한 연구

  • Kim, Jinmo (Division of Computer Engineering, Hansung University)
  • Received : 2019.02.19
  • Accepted : 2019.04.30
  • Published : 2019.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

This study compares hand-based interfaces to improve a user's virtual reality (VR) presence by enhancing user immersion in VR interactions. To provide an immersive experience, in which users can more directly control the virtual environment and objects within that environment using their hands and, to simultaneously minimize the device burden on users using immersive VR systems, we designed two experimental interfaces (hand motion recognition sensor- and controller-based interactions). Hand motion recognition sensor-based interaction reflects accurate hand movements, direct gestures, and motion representations in the virtual environment, and it does not require using a device in addition to the VR head mounted display (HMD). Controller-based interaction designs a generalized interface that maps the gesture to the controller's key for easy access to the controller provided with the VR HMD. The comparative experiments in this study confirm the convenience and intuitiveness of VR interactions using the user's hand.

본 연구는 몰입형 가상현실에서의 상호작용 과정에서 사용자의 몰입을 높여 향상된 현존감을 제공하는 손 기반 인터페이스에 대한 비교 실험을 진행한다. 몰입형 가상현실에서 사용자에게 장비의 부담을 최소화하면서 동시에 손을 보다 직접적으로 사용하여 가상 환경과 객체를 제어할 수 있는 몰입형 체험 환경을 제공하기 위하여 손 동작 인식 센서 기반 상호작용과 컨트롤러 기반 상호작용의 두 가지 실험 인터페이스를 설계한다. 손 동작 인식 센서 기반 상호작용은 VR (virtual reality) HMD (head mounted display) 외에 추가적인 장비를 사용하는 대신 정확한 손의 움직임과 직접적인 제스쳐 및 동작 표현을 가상 환경에 반영한다. 컨트롤러 기반 상호작용은 VR HMD와 함께 제공되는 컨트롤러를 쉽게 활용할 수 있도록 손의 제스쳐와 컨트롤러의 키를 매핑시킨 일반화된 인터페이스를 설계한다. 이러한 과정을 통해 본 연구는 손을 활용한 가상현실에서의 상호작용이 사용자의 편리함과 직관성에 측면에서 현존감에 미치는 영향을 비교 실험을 통해 확인한다.

Keywords

References

  1. H. Joo, T. Simon, and Y. Sheikh, "Total capture: A 3d deformation model for tracking faces, hands, and bodies," in Proceedings of The 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ser. CVPR '18, vol. abs/1801.01615. Washington, DC, USA: IEEE Computer Society, 18-22 June 2018, pp. 8320-8329. [Online]. Available: http://arxiv.org/abs/1801.01615
  2. M. Kim, J. Lee, C. Jeon, and J. Kim, "A study on interaction of gaze pointer-based user interface in mobile virtual reality environment," Symmetry, vol. 9, no. 9, p. 189, 2017. [Online]. Available: http://www.mdpi.com/2073-8994/9/9/189 https://doi.org/10.3390/sym9090189
  3. S. Marwecki, M. Brehm, L. Wagner, L.-P. Cheng, F. F. Mueller, and P. Baudisch, "Virtualspace - overloading physical space with multiple virtual reality users," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ser. CHI '18. New York, NY, USA: ACM, 21-26 April 2018, pp. 241:1-241:10. [Online]. Available: http://doi.acm.org/10.1145/3173574.3173815
  4. A. Jayasiri, S. Ma, Y. Qian, K. Akahane, and M. Sato, "Desktop versions of the string-based haptic interface - spidar," in 2015 IEEE Virtual Reality (VR). IEEE, 23-27 March 2015, pp. 199-200.
  5. C. Pacchierotti, F. Chinello, M. Malvezzi, L. Meli, and D. Prattichizzo, "Two finger grasping simulation with cutaneous and kinesthetic force feedback," in Haptics: Perception, Devices, Mobility, and Communication (EuroHaptics 2012), P. Isokoski and J. Springare, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 13-15 June 2012, pp. 373-382.
  6. P. Lindemann and G. Rigoll, "A diminished reality simulation for driver-car interaction with transparent cockpits," in 2017 IEEE Virtual Reality (VR). IEEE, 18-22 March 2017, pp. 305-306.
  7. T. Pfeiffer, Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces. Aachen, Germany: Shaker Verlag GmbH, December 2011.
  8. T. Pfeiffer, "Using virtual reality technology in linguistic research," in 2012 IEEE Virtual Reality Workshops (VRW). IEEE, 4-8 March 2012, pp. 83-84.
  9. R. Tesic and P. Banerjee, "Exact collision detection using virtual objects in virtual reality modeling of a manufacturing process," Journal of Manufacturing Systems, vol. 18, no. 5, pp. 367-376, 1999. https://doi.org/10.1016/S0278-6125(00)87639-6
  10. Bharath, V. G. and Patil, Rajashekar, "Solid modelling interaction with sensors for virtual reality welding," MATEC Web Conf., vol. 144, p. 01008, 2018. [Online]. Available: https://doi.org/10.1051/matecconf/201714401008
  11. A. Henrysson, M. Billinghurst, and M. Ollila, "Virtual object manipulation using a mobile phone," in Proceedings of the 2005 International Conference on Augmented Teleexistence, ser. ICAT '05. New York, NY, USA: ACM, 5-8 December 2005, pp. 164-171. [Online]. Available: http://doi.acm.org/10.1145/1152399.1152430
  12. S. Han and J. Kim, "A study on immersion of hand interaction for mobile platform virtual reality contents," Symmetry, vol. 9, no. 2, p. 22, 2017. https://doi.org/10.3390/sym9020022
  13. W. Zhao, J. Chai, and Y.-Q. Xu, "Combining marker-based mocap and rgb-d camera for acquiring high-fidelity hand motion data," in Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ser. SCA '12. Aire-la-Ville, Switzerland, Switzerland: Eurographics Association, 29-31 July 2012, pp. 33-42.
  14. Y. Visell, F. Fontana, B. Giordano, R. Nordahl, S. Serafin, and R. Bresin, "Sound design and perception in walking interactions," International Journal of Human-Computer Studies, vol. 67, no. 11, pp. 947-959, 2009, special issue on Sonic Interaction Design. https://doi.org/10.1016/j.ijhcs.2009.07.007
  15. Y. Visell, A. Law, J. Ip, S. Smith, and J. R. Cooperstock, "Interaction capture in immersive virtual environments via an intelligent floor surface," in 2010 IEEE Virtual Reality Conference (VR). IEEE, 20-24 March 2010, pp. 313-314.
  16. J. Lee, K. Jeong, and J. Kim, "Mave: Maze-based immersive virtual environment for new presence and experience," Computer Animation and Virtual Worlds, vol. 28, no. 3-4, p. e1756, 2017. https://doi.org/10.1002/cav.1756
  17. K. Minamizawa, S. Fukamachi, H. Kajimoto, N. Kawakami, and S. Tachi, "Gravity grabber: Wearable haptic display to present virtual mass sensation," in ACM SIGGRAPH 2007 Emerging Technologies, ser. SIGGRAPH '07. New York, NY, USA: ACM, 5-9 August 2007. [Online]. Available: http://doi.acm.org/10.1145/1278280.1278289
  18. M. Oyarzabal, M. Ferre, S. Cobos, M. Monroy, J. Barrio, and J. Ortego, "Multi-finger haptic interface for collaborative tasks in virtual environments," in Human-Computer Interaction. Interaction Platforms and Techniques (HCI 2007), J. A. Jacko, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 22-27 July 2007, pp. 673-680.
  19. K. Koyanagi, Y. Fujii, and J. Furusho, "Development of vr-stef system with force display glove system," in Proceedings of the 2005 International Conference on Augmented Tele-existence, ser. ICAT '05. New York, NY, USA: ACM, 5-8 December 2005, pp. 91-97. [Online]. Available: http://doi.acm.org/10.1145/1152399.1152417
  20. M. D. Rinderknecht, Y. Kim, L. Santos-Carreras, H. Bleuler, and R. Gassert, "Combined tendon vibration and virtual reality for post-stroke hand rehabilitation," in 2013 World Haptics Conference (WHC). IEEE, 14-17 April 2013, pp. 277- 282.
  21. I. Choi, E. W. Hawkes, D. L. Christensen, C. J. Ploch, and S. Follmer, "Wolverine: A wearable haptic interface for grasping in virtual reality," in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 9-14 Oct 2016, pp. 986-993.
  22. M. Kim, C. Jeon, and J. Kim, "A study on immersion and presence of a portable hand haptic system for immersive virtual reality," Sensors, vol. 17, no. 5, p. 1141, 2017. [Online]. Available: http://www.mdpi.com/1424-8220/17/5/1141 https://doi.org/10.3390/s17051141
  23. M. Slater and M. V. Sanchez-Vives, "Enhancing our lives with immersive virtual reality," Frontiers in Robotics and AI, vol. 3, p. 74, 2016.
  24. J. Lee, M. Kim, and J. Kim, "A study on immersion and vr sickness in walking interaction for immersive virtual reality applications," Symmetry, vol. 9, no. 5, p. 78, 2017. https://doi.org/10.3390/sym9050078
  25. K. Jeong, J. Lee, and J. Kim, "A study on new virtual reality system in maze terrain," International Journal of Human-Computer Interaction, vol. 34, no. 2, pp. 129-145, 2018. [Online]. Available: https://doi.org/10.1080/10447318.2017.1331535
  26. K. Jeong, S. Han, D. Lee, and J. Kim, "A Study on Virtual Reality Techniques for Immersive Traditional Fairy Tale Contents Production," Journal of the Korea Computer Graphics Society, vol. 22, pp. 43-52, 2016. [Online]. Available: http://www.dbpia.co.kr/Article/NODE06716018 https://doi.org/10.15701/kcgs.2016.22.3.43
  27. C. Jeon, M. Kim, J. Lee, and J. Kim, "A Study on Production Pipeline for Third Person Virtual Reality Contents Based on Hand Interface," Journal of the Korea Computer Graphics Society, vol. 23, pp. 9-17, 2017. [Online]. Available: http://www.dbpia.co.kr/Article/NODE07203594
  28. C. D. Metcalf, S. V. Notley, P. H. Chappell, J. H. Burridge, and V. T. Yule, "Validation and application of a computational model for wrist and hand movements using surface markers," IEEE Transactions on Biomedical Engineering, vol. 55, no. 3, pp. 1199-1210, March 2008. https://doi.org/10.1109/TBME.2007.908087
  29. C. Carvalheiro, R. Nobrega, H. da Silva, and R. Rodrigues, "User redirection and direct haptics in virtual environments," in Proceedings of the 2016 ACM on Multimedia Conference, ser. MM '16. New York, NY, USA: ACM, 15-19 October 2016, pp. 1146-1155.
  30. B. G. Witmer, C. J. Jerome, and M. J. Singer, "The factor structure of the presence questionnaire," Presence: Teleoper. Virtual Environ., vol. 14, no. 3, pp. 298-312, jun 2005. https://doi.org/10.1162/105474605323384654

Cited by

  1. Design of Handwriting-based Text Interface for Support of Mobile Platform Education Contents vol.27, pp.5, 2021, https://doi.org/10.15701/kcgs.2021.27.5.81