Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

A study on visuomotor and visuotactile synchronization in full body ownership illusion with virtual avatars

가상 아바타와의 전신 몸 소유감에서 시각-운동 및 시각-촉각 동기화에 관한 연구

  • 오진택 (한양대학교컴퓨터.소프트웨어학과) ;
  • 김지환 (한양대학교컴퓨터.소프트웨어학과) ;
  • 김광욱 (한양대학교컴퓨터.소프트웨어학과)
  • Received : 2022.06.09
  • Accepted : 2022.07.06
  • Published : 2022.07.26
Download PDF
⟨ Previous Next ⟩

Abstract

Body ownership illusion (BOI), which means 'the illusion that a non-bodily object feels like my body', has been actively studied since the Rubber-hand Illusion showed that a person can feel a rubber hand like one's own hand. It has been proven that BOI can be elicited for virtual bodies in virtual reality. Although many studies have been examined the effect of visuomotor (VM) and visuotactile (VT) stimuli, which are mainly used for the elicitation of BOI, there were very limited studies that delivered both stimuli to the whole body at the same time. In this paper, we investigated how each stimulus affects BOI when delivering VM and VT stimulisimultaneously to the virtual avatar, and examined user experience that appears as presence, emotion, and virtual motion sickness. The results showed that BOI was high when VM is synchronous, but there was no significant difference according to VT levels. In the case of presence, it was confirmed that VT affects only when VM is synchronous, and in the case of emotion, both VM and VT affect valence, and in the case of virtual motion sickness, a statistical difference is not found.These results suggest that overall synchrony of VM is important factor in BOI with virtual avatars, but that the matching VT affects subjective experience such as presence when VM is synchronous.

'실제 신체가 아닌 대상이 내 신체처럼 느껴지는 환상'을 의미하는 몸 소유감 환상 (Body Ownership Illusion; BOI)은 고무 손 환상 (Rubber-Hand Illusion) 실험에서 사람이 고무손을 자신의 손처럼 느낄 수 있다는 것을 보여준 뒤로 활발히 연구되어 왔으며, 가상 현실에서도 가상 신체에 대해 BOI가 형성될 수 있다는 것이 증명되었다. BOI의 형성을 위해 주로 사용된 Visuomotor(VM; 시각-운동), Visuotactile(VT; 시각-촉각) 자극이 어떤 영향을 미치는지 많은 연구가 되어왔으나 두 자극을 동시에 전신에 전달한 연구는 매우 제한적이었다. 본 연구에서는 가상 아바타와의 전신 몸 소유감에서 VM, VT 자극을 동시에 전달했을 때 각 자극이 BOI에 어떤 영향을 미치는지 연구하였고 추가적으로 임장감, 감정, 가상 멀미감으로 나타나는 사용자 경험을 연구하였다. 연구결과 BOI 측정 결과는 VM 측면에서는 자극이 일치할 때 높게 나타났으나 VT의 경우는 유의미한 차이가 나타나지 않았다. 임장감의 경우 VT는 VM이 일치하는 경우에서만 영향을 주는 것을 확인하였으며, 감정의 경우 정서가 정도에서 VM과 VT 자극이 모두 영향을 주고, 가상 멀미감의 경우 통계적인 차이를 확인하지 못하였다. 이러한 연구 결과는 가상 아바타와의 전신 몸 소유감에서 전반적으로 VM의 일치도가 중요한 요인이지만, VM이 일치되는 경우 VT의 일치가 임장감과 같은 주관적 경험에 영향을 줌을 시사한다.

Keywords

Acknowledgement

이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. 2021R1A2C2013479). 이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2021-0-00590, 대규모 노드에서 블록단위의 효율적인 거래 확정을 위한 최종성 보장 기술개발). Correspondence to K. Kim(kenny@hanyang.ac.kr)

References

  1. Botvinick, M., & Cohen, J., Rubber hands 'feel'touch that eyes see. Nature, 391(6669), 756-756, 1998. https://doi.org/10.1038/35784
  2. Slater, M., Perez Marcos, D., Ehrsson, H., & Sanchez-Vives, M. V., Towards a digital body: the virtual arm illusion. Frontiers in human neuroscience, 2, 6, 2008. https://doi.org/10.3389/neuro.09.006.2008
  3. Sanchez-Vives, M. V., Spanlang, B., Frisoli, A., Bergamasco, M., & Slater, M., Virtual hand illusion induced by visuomotor correlations. PloS one, 5(4), e10381, 2010. https://doi.org/10.1371/journal.pone.0010381
  4. Petkova, V. I., & Ehrsson, H. H., If I were you: perceptual illusion of body swapping. PloS one, 3(12), e3832, 2008. https://doi.org/10.1371/journal.pone.0003832
  5. Freeman, G., & Maloney, D., Body, avatar, and me: The resentation and perception of self in social virtual reality. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3), 1-27, 2021.
  6. Steed, A., Pan, Y., Zisch, F., & Steptoe, W., The impact of a self-avatar on cognitive load in immersive virtual reality. In 2016 IEEE virtual reality (VR) (pp. 67-76). IEEE, 2016, March.
  7. Khundam, C., First person movement control with palm normal and hand gesture interaction in virtual reality. In 2015 12th International Joint Conference on Computer Science and Software Engineering (JCSSE) (pp. 325-330). IEEE, 2015, July.
  8. Slater, M., Pertaub, D. P., & Steed, A., Public speaking in virtual reality: Facing an audience of avatars. IEEE Computer Graphics and Applications, 19(2), 6-9, 1999. https://doi.org/10.1109/38.749116
  9. Maselli, A., & Slater, M., The building blocks of the full body ownership illusion. Frontiers in human neuroscience, 7, 83, 2013. https://doi.org/10.3389/fnhum.2013.00083
  10. Kokkinara, E., & Slater, M., Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion. Perception, 43(1), 43-58, 2014. https://doi.org/10.1068/p7545
  11. Kim, C. S., Jung, M., Kim, S. Y., & Kim, K., Controlling the sense of embodiment for virtual avatar applications: methods and empirical study. JMIR Serious Games, 8(3), e21879, 2020. https://doi.org/10.2196/21879
  12. Jun, J., Jung, M., Kim, S. Y., & Kim, K., Full-body ownership illusion can change our emotion. In Proceedings of the 2018 CHI conference on human factors in computing systems(pp. 1-11), 2018, April.
  13. Kondo, R., Sugimoto, M., Minamizawa, K., Hoshi, T., Inami, M., & Kitazaki, M., Illusory body ownership of an invisible body interpolated between virtual hands and feet via visual-motor synchronicity. Scientific reports, 8(1), 1-8, 2018.
  14. Ehrsson, H. H., Multisensory processes in body ownership. Multisensory perception, 179-200, 2020.
  15. Limanowski, J., Lutti, A., & Blankenburg, F., The extrastriate body area is involved in illusory limb ownership. Neuroimage, 86, 514-524, 2014. https://doi.org/10.1016/j.neuroimage.2013.10.035
  16. Normand, J. M., Giannopoulos, E., Spanlang, B., & Slater, M., Multisensory stimulation can induce an illusion of larger belly size in immersive virtual reality. PloS one, 6(1), e16128, 2011. https://doi.org/10.1371/journal.pone.0016128
  17. Kilteni, K., Bergstrom, I., & Slater, M., Drumming in immersive virtual reality: the body shapes the way we play. IEEE transactions on visualization and computer graphics, 19(4), 597-605, 2013. https://doi.org/10.1109/TVCG.2013.29
  18. Perez-Marcos, D., Sanchez-Vives, M. V., & Slater, M., Is my hand connected to my body? The impact of body continuity and arm alignment on the virtual hand illusion. Cognitive neurodynamics, 6(4), 295-305, 2012. https://doi.org/10.1007/s11571-011-9178-5
  19. Slater, M., & Wilbur, S., A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 6(6), 603-616, 1997. https://doi.org/10.1162/pres.1997.6.6.603
  20. Kim, S. Y., Park, H., Jung, M., & Kim, K., Impact of Body Size Match to an Avatar on the Body Ownership Illusion and User's Subjective Experience. Cyberpsychology, Behavior, and Social Networking, 23(4), 234-241, 2020. https://doi.org/10.1089/cyber.2019.0136
  21. Kuschel, M., Freyberger, F., Buss, M., & Farber, B.,A presence measure for virtual reality and telepresence based on multimodal conflicts. In Proceedings of PRESENCE 2007: The 10th Annual International Workshop on Presence, 2007.
  22. Wolf, D., Rietzler, M., Hnatek, L., & Rukzio, E., Face/on: multimodal haptic feedback for head-mounted displays in virtual reality. IEEE transactions on visualization and computer graphics, 25(11), 3169-3177, 2019. https://doi.org/10.1109/tvcg.2019.2932215
  23. Williams, B., Garton, A. E., & Headleand, C. J., Exploring visuohaptic feedback congruency in virtual reality. In 2020 International Conference on Cyberworlds (CW) (pp. 102-109). IEEE, 2020, September.
  24. Bradley, M. M., & Lang, P. J., Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry, 25(1), 49-59, 1994. https://doi.org/10.1016/0005-7916(94)90063-9
  25. Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G., Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology, 3(3), 203-220, 1993. https://doi.org/10.1207/s15327108ijap0303_3
  26. LaViola Jr, J. J., A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin, 32(1), 47-56, 2000. https://doi.org/10.1145/333329.333344
  27. Tamaki, E., Miyaki, T., & Rekimoto, J., PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp. 543-552), 2011, May.
  28. Caserman, P., Krug, C., & Gobel, S., Recognizing Full-Body Exercise Execution Errors Using the Teslasuit. Sensors, 21(24), 8389, 2021. https://doi.org/10.3390/s21248389
  29. Banakou, D., & Slater, M., Body ownership causes illusory self-attribution of speaking and influences subsequent real speaking. Proceedings of the National Academy of Sciences, 111(49), 17678-17683, 2014. https://doi.org/10.1073/pnas.1414936111
  30. Witmer, B. G., & Singer, M. J., Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225-240, 1998. https://doi.org/10.1162/105474698565686
  31. O'Kane, S. H., & Ehrsson, H. H., The contribution of stimulating multiple body parts simultaneously to the illusion of owning an entire artificial body. Plos one, 16(1), e0233243, 2021. https://doi.org/10.1371/journal.pone.0233243
  32. Auda, J., Pascher, M., & Schneegass, S., Around the (Virtual) World: Infinite Walking in Virtual Reality Using Electrical Muscle Stimulation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(pp. 1-8), 2019, May.
  33. Khamis, M., Schuster, N., George, C., & Pfeiffer, M., Electrocutscenes: Realistic haptic feedback in cutscenes of virtual reality games using electric muscle stimulation. In 25th ACM Symposium on Virtual Reality Software and Technology (pp. 1-10), 2019, November.
  34. Lopes, P., You, S., Cheng, L. P., Marwecki, S., & Baudisch, P., Providing haptics to walls & heavy objects in virtual reality by means of electrical muscle stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(pp. 1471-1482), 2017, May.
  35. Park, W., Kim, J., & Lee, J.,A Study on the Design and Effect of Feedback for Virtual Reality Exercise Posture Training. Journal of the Korea Computer Graphics Society, 26(3), 79-86, 2020. https://doi.org/10.15701/kcgs.2020.26.3.79