Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Effect of Simulator Sickness Caused by Head-mounted Display on the Stability of the Pupillary Rhythm

머리착용 디스플레이에 의해 유발된 멀미 증상이 동공 리듬의 안정성에 미치는 영향

  • Park, Sangin (Industry-Academy Cooperation Foundation, Sangmyung University) ;
  • Lee, Don Won (Department of Emotion Engineering, Sangmyung University) ;
  • Mun, Sungchul (CJ Hello Future Engine Lab., CJ Hello) ;
  • Kim, Hong-Ik (Technology Officer, CJ Hello) ;
  • Whang, Mincheol (Department of Intelligent Engineering Informations for Human, Sangmyung University)
  • 박상인 (상명대학교 산학협력단) ;
  • 이동원 (상명대학교 감성공학과) ;
  • 문성철 (CJ 헬로 Future Engine Lab.) ;
  • 김홍익 (CJ 헬로 기술담당) ;
  • 황민철 (상명대학교 미래융합공학대학 휴먼지능정보공학과)
  • Received : 2018.08.27
  • Accepted : 2018.12.11
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

The aim of this study is to determine the effect of motion sickness on pupil rhythm. Sixteen volunteers of both genders (8 male, 8 female, mean age $25.67{\pm}2.43$ years) experienced VR contents in both 2D and HMD versions for 15 minutes, and their pupillary rhythms were compared. The irregular pattern of the pupillary rhythms, as demonstrated by increasing mean pupil diameter (mPD) and standard deviation of the pupil diameter (sPD), revealed motion sickness after experiencing HMD condition. The pupillary response is strongly related to the cognitive load, and the motion sickness can be interpreted as a change in the cognitive load caused by the increasing volume of visual information that must be processed and the conflict or inconsistency between different sensory modalities. The method proposed in this study could be a non-contact measurement method for the monitoring of motion sickness using a web-camera rather than previous sensor-based methods.

본 연구는 멀미증상이 동공 리듬에 미치는 영향을 확인하는 것을 그 목적으로 하였다. 본 연구는 2D와 HMD를 통해 동일한 가상현실 콘텐츠를 실험참가자에게 15분 동안 경험하게 한 이후에 동공반응을 비교하였고 본 실험에는 16명(남자 8명, 여자 8명, 평균 나이: $25.67{\pm}2.43$세)의 피험자가 참여하였다. 연구 결과, 영상멀미를 경험한 이후에 동공 리듬이 불규칙한 패턴을 나타내었고 이것은 동공 크기(mPD) 및 동공 리듬의 편차(sPD)가 증가하는 결과로 확인되었다. 이러한 동공 반응의 결과는 인지부하와 밀접한 관련이 있고 영상멀미를 처리해야 할 시각 정보량의 증가 및 감각정보들 사이의 충돌 혹은 불일치로 인한 인지부하 현상으로 해석이 가능하다. 본 연구에서 제안된 방법은 기존의 센서 기반의 평가 방법에 비해 멀미 증상을 카메라 기반의 비접촉식으로 측정할 수 있는 가능성을 확인하였다.

Keywords

References

  1. Andreassi, J. L. (2013). Psychophysiology: Human behavior & physiological response. Psychology Press.
  2. Bailenson, J., Patel, K., Nielsen, A., Bajscy, R., Jung, S. H., & Kurillo, G. (2008). The effect of interactivity on learning physical actions in virtual reality. Media Psychology, 11(3), 354-376. DOI: 10.1080/15213260802285214
  3. Beatty, J. (1988). Pupillometric signs of selective attention in man. Neurophysiology and psychophysiology: Experimental and clinical applications, 138-143.
  4. Biocca, F., & Delaney, B. (1995). Immersive virtual reality technology. Communication in the Age of Virtual Reality, 15, 57-124.
  5. Bouchard, S., Robillard, G., Renaud, P., & Bernier, F. (2011). Exploring new dimensions in the assessment of virtual reality induced side effects. Journal of Computer and Information Technology, 1(3), 20-32.
  6. Buckner, R. L. (2013). The brain's default network: origins and implications for the study of psychosis. Dialogues in Clinical Neuroscience, 15(3), 351-358.
  7. Carnegie, K., & Rhee, T. (2015). Reducing visual discomfort with HMDs using dynamic depth of field. IEEE Computer Graphics and Applications, 35(5), 34-41. DOI: 10.1109/MCG.2015.98
  8. Chardonnet, J. R., Mirzaei, M. A., & Merienne, F. (2017). Features of the Postural Sway Signal as Indicators to Estimate and Predict Visually Induced Motion Sickness in Virtual Reality. International Journal of Human-Computer Interaction, 1-15. DOI: 10.1080/10447318.2017.1286767
  9. Chen, S., & Epps, J. (2014). Using task-induced pupil diameter and blink rate to infer cognitive load. Human-Computer Interaction, 29(4), 390-413. DOI: 10.1080/07370024.2014.892428
  10. Chuang, S. W., Chuang, C. H., Yu, Y. H., King, J. T., & Lin, C. T. (2016). EEG alpha and gamma modulators mediate motion sickness-related spectral responses. International Journal of Neural Systems, 25(2), 14. DOI: 10.1142/S0129065716500076
  11. Daugman, J. (2004). Iris recognition border-crossing system in the UAE. International Airport Review, 8(2), 49-53.
  12. Dunnett, C. W. (1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association, 50(272), 1096-1121. DOI: 10.1080/01621459.1955.10501294
  13. Ebitz, R. B., & Platt, M. L. (2015). Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupillinked arousal. Neuron, 85(3), 628-640. DOI: 10.1016/j.neuron.2014.12.053
  14. Geva, R., Zivan, M., Warsha, A., & Olchik, D. (2013). Alerting, orienting or executive attention networks: differential patters of pupil dilations. Frontiers in Behavioral Neuroscience, 7, 145(1)-145(11). DOI: 10.3389/fnbeh.2013.00145
  15. Golding, J. F. (1998). Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. Brain Research Bulletin, 47(5), 507-516. DOI: 10.1016/S0361-9230(98)00091-4
  16. Hong, L., Walz, J. M., & Sajda, P. (2014). Your eyes give you away: Prestimulus changes in pupil diameter correlate with poststimulus task-related EEG dynamics. PloS One, 9(3), e91321. DOI: 10.1371/journal.pone.0091321
  17. Huck, S. W., Cormier, W. H., & Bounds, W. G. (1974). Reading statistics and research (pp. 74-102). New York: Harper & Row.
  18. Jeong, S. H. (2010). Motion sickness. Korean Journal of Aerospace and Environmental Medicine, 19(3), 87-90.
  19. Joshi, S., Li, Y., Kalwani, R. M., & Gold, J. I. (2016). Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron, 89(1), 221-234. DOI: 10.1016/j.neuron.2015.11.028
  20. Junior, E. D. D. S., Da Silva, A. V., Da Silva, K. R., Haemmerle, C. A., Batagello, D. S., Da Silva, J. M., Lima, L. B., Da Silva, R. J., Diniz, G. B., Sita, L. V., Elias, C. F., & Bittencourt, J. C. (2015). The centrally projecting Edinger–Westphal nucleus-I: Efferents in the rat brain. Journal of Chemical Neuroanatomy, 68, 22-38. DOI: 10.1016/j.jchemneu.2015.07.002
  21. Just, M. A., & Carpenter, P. A. (1993). The intensity dimension of thought: pupillometric indices of sentence processing. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Experimentale, 47(2), 310-339. DOI: 10.1037/h0078820
  22. Just, M. A., Carpenter, P. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1996). Brain activation modulated by sentence comprehension. Science, 114-116. DOI: 10.1126/science.274.5284.114
  23. Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3(3), 203-220. DOI: 10.1207/s15327108ijap0303_3
  24. Kennedy, R. S., Drexler, J., & Kennedy, R. C. (2010). Research in visually induced motion sickness. Applied Ergonomics, 41(4), 494-503. DOI: 10.1016/j.apergo.2009.11.006
  25. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68(3), 350-386. DOI: 10.3102/00346543068003350
  26. Kesim, M., & Ozarslan, Y. (2012). Augmented reality in education: current technologies and the potential for education. Procedia-Social and Behavioral Sciences, 47, 297-302. DOI: 10.1016/j.sbspro.2012.06.654
  27. Kim, Y. Y., Kim, H. J., Kim, E. N., Ko, H. D., & Kim, H. T. (2005). Characteristic changes in the physiological components of cybersickness. Psychophysiology, 42(5), 616-625. DOI: 10.1111/j.1469-8986.2005.00349.x
  28. Kiryu, T., Tada, G., Toyama, H., & Iijima, A. (2008). Integrated evaluation of visually induced motion sickness in terms of autonomic nervous regulation. In Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE, 4597-4600. DOI: 10.1109/IEMBS.2008.4650237
  29. Klingner, J., Kumar, R., & Hanrahan, P. (2008, March). Measuring the task-evoked pupillary response with a remote eye tracker. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications, 69-72. DOI: 10.1145/1344471.1344489
  30. Kozicz, T., Bittencourt, J. C., May, P. J., Reiner, A., Gamlin, P. D., Palkovits, M., ... & Ryabinin, A. E. (2011). The Edinger‐Westphal nucleus: A historical, structural, and functional perspective on a dichotomous terminology. Journal of Comparative Neurology, 519(8), 1413-1434. DOI: 10.1002/cne.22580
  31. Krause, C. M., Pesonen, M., Haarala Bjornberg, C., & Hamalainen, H. (2007). Effects of pulsed and continuous wave 902 MHz mobile phone exposure on brain oscillatory activity during cognitive processing. Bioelectromagnetics: Journal of the Bioelectromagnetics Society, The Society for Physical Regulation in Biology and Medicine. The European Bioelectromagnetics Association, 28(4), 296-308. DOI: 10.1002/bem.20300
  32. Lambooij, M.T.M., Fortuin, M., Heynderick, I., Ijsselsteijn, W. (2009). Visual discomfort and visual fatigue of stereoscopic displays: a review. Journal of Imaging Science and Technology, 53(3), 1-14. DOI: 10.2352/J.ImagingSci.Technol.2009.53.3.030201
  33. Lee, E. C., Park, K. R., Whang, M., & Min, K. (2009). Measuring the degree of eyestrain caused by watching LCD and PDP devices. International Journal of Industrial Ergonomics, 39(5), 798-806. DOI: 10.1016/j.ergon.2009.02.008
  34. Merhi, O., Faugloire, E., Flanagan, M., & Stoffregen, T. A. (2007). Motion sickness, console video games, and head-mounted displays. Human Factors, 49(5), 920-934. DOI: 10.1518/001872007X230262
  35. Mun, S., Park, M. C., Park, S., & Whang, M. (2012). SSVEP and ERP measurement of cognitive fatigue caused by stereoscopic 3D. Neuroscience Letters, 525(2), 89-94. DOI: 10.1016/j.neulet.2012.07.049
  36. Mun, S., Kim, E. S., & Park, M. C. (2014). Effect of mental fatigue caused by mobile 3D viewing on selective attention: An ERP study. International Journal of Psychophysiology, 94(3), 373-381. DOI: 10.1016/j.ijpsycho.2014.08.1389
  37. Nalivaiko, E., Davis, S. L., Blackmore, K. L., Vakulin, A., & Nesbitt, K. V. (2015). Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Physiology & Behavior, 151, 583-590. DOI: 10.1016/j.physbeh.2015.08.043
  38. Naqvi, S. A. A., Badruddin, N., Malik, A. S., Hazabbah, W., & Abdullah, B. (2013). Does 3D produce more symptoms of visually induced motion sickness?. In Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE, 6405-6408. DOI: 10.1109/EMBC.2013.6611020
  39. Oman, C. M. (1990). Motion sickness: a synthesis and evaluation of the sensory conflict theory. Canadian Journal of Physiology and Pharmacology, 68(2), 294-303. DOI: 10.1139/y90-044
  40. Ong, S. K., & Nee, A. Y. C. (2013). Virtual and augmented reality applications in manufacturing. Springer Science & Business Media.
  41. Park, S., Won, M. J., Mun, S., Lee, E. C., & Whang, M. (2014). Does visual fatigue from 3D displays affect autonomic regulation and heart rhythm?. International Journal of Psychophysiology, 92(1), 42-48. DOI: 10.1016/j.ijpsycho.2014.02.003
  42. Park, S., Won, M. J., Lee, E. C., Mun, S., Park, M. C., & Whang, M. (2015). Evaluation of 3D cognitive fatigue using heart–brain synchronization. International Journal of Psychophysiology, 97(2), 120-130. DOI: 10.1016/j.ijpsycho.2015.04.006
  43. Park, S., Won, M. J., Lee, D. W., & Whang, M. (2018). Non-contact measurement of heart response reflected in human eye. International Journal of Psychophysiology, 123, 179-198. DOI: 10.1016/j.ijpsycho.2017.07.014
  44. Park, S., & Whang, M. (2018). Infrared camera-based noncontact measurement of brain activity from pupillary rhythms. Frontiers in Physiology, 9, 1400. DOI: 10.3389/fphys.2018.01400
  45. Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-computer Studies, 59(1-2), 185- 198. DOI: 10.1016/S1071-5819(03)00017-X
  46. Raajan, N. R., Suganya, S., Priya, M. V., Ramanan, S. V., Janani, S., Nandini, N. S., Hemanand, R., & Gayathri, S. (2012). Augmented reality based virtual reality. Procedia Engineering, 38, 1559-1565. DOI: 10.1016/j.proeng.2012.06.191
  47. Ryan, M. L. (1999). Immersion vs. interactivity: Virtual reality and literary theory. SubStance, 28(2), 110-137. DOI: 10.1353/sub.1999.0015
  48. Siegle, G. J., Steinhauer, S. R., & Thase, M. E. (2004). Pupillary assessment and computational modeling of the Stroop task in depression. International Journal of Psychophysiology, 52(1), 63-76. DOI: 10.1016/j.ijpsycho.2003.12.010
  49. Tanaka, M., Ishii, A., & Watanabe, Y. (2014). Neural effects of mental fatigue caused by continuous attention load: a magnetoencephalography study. Brain Research, 1561, 60-66. DOI: 10.1016/j.brainres.2014.03.009
  50. Toschi, N., Kim, J., Sclocco, R., Duggento, A., Barbieri, R., Kuo, B., & Napadow, V. (2017). Motion sickness increases functional connectivity between visual motion and nausea-associated brain regions. Autonomic Neuroscience, 202, 108-113. DOI: 10.1016/j.autneu.2016.10.003
  51. Van den Brink, R. L., Murphy, P. R., & Nieuwenhuis, S. (2016). Pupil diameter tracks lapses of attention. PloS One, 11(10), e0165274. DOI: 10.1371/journal.pone.0165274
  52. Van Krevelen, D. W. F., & Poelman, R. (2010). A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality, 9(2), 1-20.
  53. Van Steenbergen, H., & Band, G. P. (2013). Pupil dilation in the Simon task as a marker of conflict processing. Frontiers in Human Neuroscience, 7, 215. DOI: 10.3389/fnhum.2013.00215
  54. Wang, Y. (2013). An investigation of visual field test parameters in glaucoma, patterns of visual field loss in diabetics and multispectral imaging of the optic nerve head in glaucoma (Doctoral dissertation, The University of Manchester (United Kingdom)).
  55. Yokota, Y., Aoki, M., Mizuta, K., Ito, Y., & Isu, N. (2005). Motion sickness susceptibility associated with visually induced postural instability and cardiac autonomic responses in healthy subjects. Acta Oto-laryngologica, 125(3), 280-285. DOI: 10.1080/00016480510003192