Journal of the Architectural Institute of Korea (대한건축학회논문집)

Architectural Institute of Korea (대한건축학회)
권호 표지
DOI QR코드

DOI QR Code

Development of a Prediction Model for EEG-based Relaxation-arousal State of Users Experiencing a Virtual Reality Space

가상현실 공간 체험자의 뇌파(EEG) 기반 이완-각성 상태 예측 모델 개발

  • Kim, Sang-Hee (A3 Architectural Research Institute, Kyung-pook National University) ;
  • Choo, Seung-Yeon (Dept. of Architectural Engineering, Kyung-pook National University)
  • Received : 2022.07.27
  • Accepted : 2022.10.24
  • Published : 2022.11.30
⟨ Previous Next ⟩

Abstract

This study was carried out to develop a model that can predict a user's relaxation-arousal state by using electroencephalogram (EEG) data and machine learning algorithms of users experiencing a virtual reality space. Specific ways were proposed to improve the prediction accuracy of this model. Upon learning about this model, the prediction performance was compared while changing the hyperparameter conditions of each model using supervised learning-based machine learning models suitable for the development of predictive models known as the random forest, support vector machine, and artificial neural network algorithms. As a result, the random forest model had the highest prediction accuracy when there were 300 trees, the support vector machine model when a sigmoid kernel was applied, and the artificial neural network model when there were five hidden layers; these results confirmed that each optimal parameter condition could be met. Each model was learned by applying the feature extraction method suggested in feature engineering to derive an improvement method in the prediction performance of each model. The results revealed that when the frequency-specific statistics and filtering-based feature extraction method was applied, the prediction performance improved in the random forest and artificial neural network models. Additionally, it was shown that the machine learning models that could best predict the relaxation-arousal state from the EEG data of users experiencing a virtual reality space was the artificial neural network model with five hidden layers applied with the frequency-specific statistics and filtering-based feature processing method; its predictive accuracy was 70.21%. The results of this study could be useful basic data to implement an automated system that evaluates the design drafts of a healing space by utilizing virtual reality and EEG data.

Keywords

Acknowledgement

이 논문은 2021년 정부(교육부)의 재원으로 한국연구재단의 지원을 받아 수행된 기초연구사업임(NO.2021R1A6A3A01086574).

References

  1. Aguinaga, A., & Ramirez, M. (2018). Emotional states recognition, implementing a low computational complexity strategy. Health Informatics Journal, 24(2), 146-170. https://doi.org/10.1177/1460458216661862
  2. Altaf, E., Aletheia, I., & Esther, M. (2020). Healing spaces: designing physical environments to optimize health, wellbeing, and performance, International Journal of Environmental Research and Public Health, 17(4), 1155. https://doi.org/10.3390/ijerph17041155
  3. Atkinson, J., & Campos, D. (2016). Improving BCI-based emotion recognition by combining EEG feature selection andkernel classifers, Expert Systems with Applications, 47, 35-41. https://doi.org/10.1016/j.eswa.2015.10.049
  4. Baker, C. (2021). Emotional Responses in Virtual Reality Environments, Ph.D. Dissertation, Liverpool John Moores University.
  5. Banaei, M., Hatami, J., Yazdanfar, A., & Gramann, K. (2017). Walking through architectural spaces: the impact of interior forms on human brain dynamics, Frontiers in Human Neuroscience, 477.
  6. Balan, O., Moise, G., Moldoveanu, A., Leordeanu, M., & Moldoveanu, F. (2019). Fear level classification based on emotional dimensions and machine learning techniques, Sensors, 19(7), 1738. https://doi.org/10.3390/s19071738
  7. Benjamin, J., Stephen, D., Karen, J., Joao, J., Ian, C., Maria, W., & Simon, Ha. (2019). Alpha/beta power decreases track the fidelity of stimulus-specific information, eLife, 8, e49562. https://doi.org/10.7554/eLife.49562
  8. Berlyne, D. (1960). Toward a Theory of Exploratory Behavior: I. Arousal and Drive, New York, McGraw-Hill Book Company, 163-192.
  9. Chanel, G., Kronegg, J., Grandjean, D., & Pun, T. (2006). Emotion assessment: arousal evaluation using EEG's and peripheral physiological signals, In international workshop on multimedia content representation, Classification and Security, 530-537.
  10. Chang, S. W., Dong, W. H., & Jun, H. J. (2018). An EEG-based deep neural network classification model for recognizing emotion of users in early phase of design, Journal of the Architectural Institute of Korea Planning & Design, 34(12), 85-94.
  11. Choi, H. W., & Kim, H. J. (2019). Meta-feature engineering for machine learning-based automated data visualization, The Journal of Korean Institute of Communications and Information Sciences, 44(9), 1788-1797. https://doi.org/10.7840/kics.2019.44.9.1788
  12. Choi, K. T. (2017). Image recognition and clustering for virtual reality based on cognitive rehabilitation contents, The Journal of Digital Contents Society, 18(7), 1249-1257. https://doi.org/10.9728/DCS.2017.18.7.1249
  13. Choi, P. S., & Min, I. S. (2018). A predictive model for the employment of college graduates using a machine learning approach, Korea Research Institute for Vocational Education & Training, 21(1), 31-54. https://doi.org/10.36907/krivet.2018.21.1.31
  14. Ciaburro, G., & Venkateswaran, B. (2017). Neural Networks with R: Smart Models using CNN, RNN, Deep Learning, and Artificial Intelligence Principles, UK, Packt Publishing Ltd, 248.
  15. Cox, T. (1978). Stress, London: Macmillan, 322.
  16. Daly, I., Malik, A., Weaver, J., Hwang, F., Nasuto, S., Williams, D., Kirke, A., & Miranda, E. (2015). Identifying music-induced emotions from EEG for use in brain-computer music interfacing, In 2015 international conference on affective computing and intelligent interaction, IEEE, 923-929.
  17. Dreyfus, S. (1990). Artificial neural networks, back propagation, and the Kelley-Bryson gradient procedure, Journal of Guidance, Control, and Dynamics, 13(5), 926-928. https://doi.org/10.2514/3.25422
  18. DuBose, J., MacAllister, L., Hadi, K., & Sakallaris, B. (2018). Exploring the concept of healing spaces. HERD: Health Environments Research & Design Journal, 11(1), 43-56. https://doi.org/10.1177/1937586716680567
  19. Ha, J. M., & Park, S. B. (2017). Assessment of color affect on the indoor color schemes and illuminance change, Journal of the Architectural Institute of Korea, Planning and Design Section, 33(10), 57-765. https://doi.org/10.5659/JAIK_PD.2017.33.3.57
  20. Hong, G. Y., & Lee, O. (2017). The influence of VR color image for color psychotherapy, The Journal of the Korea Contents Association, 17(10), 376-384. https://doi.org/10.5392/JKCA.2017.17.10.376
  21. Hwang, J. E. (2014), Familiar challenges for joint studies of neuroscience and architecture, Journal of the Architectural Institute of Korea, 58(9), 8-11.
  22. Hwang, J. H., & Park, M. S. (2018). Effect of a dual-task virtual reality program for seniors with mild cognitive impairment, Korean Journal of Clinical Laboratory Science, 50(4), 492-500. https://doi.org/10.15324/kjcls.2018.50.4.492
  23. Hwang, M. (2017). A Study on EEG Arousal Effect on Stimulation of Color, Ph.D. Dissertation, Kyungsung University, 132.
  24. Je, H. (2019). Therapeutic Effect of Interactive Experience in Virtual Garden : a Physiological Approach, Thesis, Seoul National University, 78.
  25. Jeong, M. S., Lim, T. H., & Ryu, J, H. (2021). Application of machine learning for the development of adaptive class materials, Journal of Field-based Lesson Studies, 2(1), 47-71. https://doi.org/10.22768/JFLS.2021.2.1.47
  26. Jiao, Z., Gao, X., Wang, Y., Li, J., & Xu, H. (2018). Deep convolutional neural networks for mental load classification based on EEG data, Pattern Recognition, 76, 582-595. https://doi.org/10.1016/j.patcog.2017.12.002
  27. Jin, B., Kim, G., Moore, M., & Rothenberg, L. (2021). Consumer store experience through virtual reality: its effect on emotional states and perceived store attractiveness, Fashion and Textiles, 8(1), 1-21. https://doi.org/10.1186/s40691-020-00228-3
  28. Kim, D. J., & Shin H. S. (2022). Emotional recognition technology and knowledge services, Korea Evaluation Institute of Industrial Technology, 22(5), 1-22.
  29. Kim, E. S. (2020). Effect of sensory stimulation on brain wave changes during resting-state: focus on Gender, Journal of Humanities and Social Sciences 21, 11(6), 591-606.
  30. Kim, I. H., Kang, Y. H., Hwa, N. H., & Do, K. Y. (2014). A Study on the changes of openness in relation to different window to wall ratio of buildings in coastal areas, Journal of the Regional Association of Architectural Institute of Korea, 16(3), 235-243.
  31. Kim, J. H. (2011). A study on an application of 'virtual reality therapy' concerning a technology of real-time interaction, Cartoon and Animation Studies, 22, 81-97. https://doi.org/10.7230/KOSCAS.2011.23.1.081
  32. Kim, S. H. (2021). Developing a VR-EEG-based Model for Optimizing Visual Perceptual Components of Healing Space, Ph. D. Dissertation, Kyung pook National University, 240.
  33. Kim, S. H., Lee, K. H., & Choo, S. Y. (2021). Analysis of EEG relaxation-arousal reaction to the window-To-wall ratio of individual rooms of a postpartum care center using EEG-VR, Journal of the Architectural Institute of Korea, 37(3), 63-74. https://doi.org/10.5659/JAIK.2021.37.3.63
  34. Kim, S. U., Kang, S. Y., Ji, S. Y., & Jun, H. J. (2019). Measurement of EEG and analysis of stress change in space using virtual reality-focus on the Hitler's residence, Journal of the Architectural Institute of Korea Planning & Design, 35(8), 73-79.
  35. Kim, Y. J., Shin, D. J., & Kim, J. Y. (2019). A study on the characteristics on brain wave of indoor space lighting by EEG experiment, Journal of Korean Institute of Spatial Design, 14(2), 71-80. https://doi.org/10.35216/kisd.2019.14.2.71
  36. Kumar, A., & Kumar, A. (2021). DEEPHER: human emotion recognition using an EEG-Based Deep learning network model, Engineering Proceedings, 10(1), 32.
  37. Lee, Y. (2005). Effects of Music and Musical Career on Guided Imaginary for Relaxation, Thesis, Ewha Womans University, 63.
  38. Lee, D. B., & Lee, S. G. (2014). The EEG signal based emotion recognition using the EMD, Journal of Advanced Information Technology and Convergence, 12(6), 47-53.
  39. Lee, E. J., Song, Y. S., Kim, J. H., & Oh, S. Y. (2020). An exploratory study on determinants predicting the dropout rate of 4-year universities using random forest, Journal of Educational Technology, 36(1), 191-219. https://doi.org/10.17232/KSET.36.1.191
  40. Lee, H. K. (2018). A exploratory study of the concept and theories of the healing environment for mental health enhancement, Design convergence study, 17(3), 109-123. https://doi.org/10.31678/SDC.70.7
  41. Lee, J. S., Ryu, J. S., Kim, H. N., & Lee, H. W. (2014). An analysis of human physiological responses to apply color to the indoor living space, Journal of the Korea Socity of Colour Studies, 28(1), 96-105. https://doi.org/10.17289/jkscs.28.1.201402.96
  42. Lee, S. W., Cho, H. J., & Chae, C. J. (2020). EEG signal classification based on SVM algorithm, Journal of the Korea Convergence Society, 11(2), 17-22. https://doi.org/10.15207/JKCS.2020.11.2.017
  43. Magdin, M., Balogh, Z., Reichel, J., Francisti, J., Koprda, S., & Gyorgy, M. (2021). Automatic detection and classification of emotional states in virtual reality and standard environments (LCD): Comparing valence and arousal of induced emotions, Virtual Reality, 25(4), 1029-1041. https://doi.org/10.1007/s10055-021-00506-5
  44. Milovanovic, J., Moreau, G., Siret, D., & Miguet, F. (2017). Virtual and augmented reality in architectural design and education, In 17th international conference, CAAD futures.
  45. Myung, J. Y., & Jun, H. J. (2020). Emotion classification DNN model for virtual reality based 3D space. Journal of the Architectural Institute of Korea Planning & Design, 36(4), 41-49.
  46. Park, S. J., & Chung, C. S. (2019). Forecasting ability of machine learning algorithms using high-frequency data, Futures Journal of Money & Finance, 33(4), 31-60. https://doi.org/10.21023/JMF.33.4.2
  47. Park, Y. J., Kim, G. Y., & Jang, S. W. (2013). Traffic anomaly identification using multi-class support vector machine, Journal of the Korea Academia-Industrial Cooperation Society, 14(4), 1942-1950. https://doi.org/10.5762/KAIS.2013.14.4.1942
  48. Sayorwan, W., Siripornpanich, V., Piriyapunyaporn, T., Hongratanaworakit, T., Kotchabhakdi, N., & Ruangrungsi, N. (2012). The effects of lavender oil inhalation on emotional states, autonomic nervous system, and brain electrical activity, Journal of the Medical Association of Thailand, 95(4), 598-606.
  49. Shin, Y. J., Hyoung, S. E., & Hong, J. P. (2013). A basic study on extraction of healing design elements for silver generation, Journal of Integrated Design Research, 12(4), 9-20 https://doi.org/10.21195/jidr.2013.12.4.001
  50. Son, G. Y., Lee, W. Y., Han, K. J., & Kyeong, S. H. (2020). The study of feature vector generation and emotion recognition using EEG signals, Journal of the Korean Institute of Next Generation Computing, 16(2), 72-79.
  51. Son, G. Y., Lee, W. Y., Lee, J. W., & Ki, M. S. (2022). Machine learning based human emotion state classification using EEG signals, Journal of the Korean Institute of Next Generation Computing, 18(1), 37-46.
  52. Subha, D., Joseph, P., Acharya U., & Lim, C. (2010). EEG signal analysis: a survey, Journal of Medical Systems, 34(2), 195-212. https://doi.org/10.1007/s10916-008-9231-z
  53. Suhaimi, N., Mountstephens, J., & Teo, J. (2022). A dataset for emotion recognition using virtual reality and EEG(DER-VREEG): emotional state classification using low-cost wearable VR-EEG headsets, Big Data and Cognitive Computing, 6(1), 16. https://doi.org/10.3390/bdcc6010016
  54. Tee, Y., & Siti, A. (2020). Electroencephalogram (EEG) stress analysis on alpha/beta ratio and theta/beta ratio, Indonesian, Journa of Electrical Engineering and Computer Science, 17(1), 175-182.
  55. Ulrich, R. (1984). View througt a window may influence recovery from surgery, Science, 224(4647), 420-421. https://doi.org/10.1126/science.6143402
  56. Wiem, M., & Lachiri, Z. (2017). Emotion classification in arousal valence model using MAHNOB-HCI database, International Journal of Advanced Computer Science and Applications, 8(3), 318-323.
  57. Yeo, M., & Lee, C. N. (2016). Characteristics of the tactile brainwave on the surface of interior finishing materials, Korean Institute of Interior Design Journal, 25(2), 59-69. https://doi.org/10.14774/JKIID.2016.25.2.059