Journal of the Architectural Institute of Korea Planning & Design (대한건축학회논문집:계획계)

Architectural Institute of Korea (대한건축학회)
권호 표지
DOI QR코드

DOI QR Code

Emotion Classification DNN Model for Virtual Reality based 3D Space

가상현실 기반 3차원 공간에 대한 감정분류 딥러닝 모델

  • 명지연 (한양대학교 대학원 건축학과) ;
  • 전한종 (한양대학교 건축학부)
  • Received : 2019.11.27
  • Accepted : 2020.04.13
  • Published : 2020.04.30
⟨ Previous Next ⟩

Abstract

The purpose of this study was to investigate the use of the Deep Neural Networks(DNN) model to classify user's emotions, in particular Electroencephalography(EEG) toward Virtual-Reality(VR) based 3D design alternatives. Four different types of VR Space were constructed to measure a user's emotion and EEG was measured for each stimulus. In addition to the quantitative evaluation based on EEG data, a questionnaire was conducted to qualitatively check whether there is a difference between VR stimuli. As a result, there is a significant difference between plan types according to the normalized ranking method. Therefore, the value of the subjective questionnaire was used as labeling data and collected EEG data was used for a feature value in the DNN model. Google TensorFlow was used to build and train the model. The accuracy of the developed model was 98.9%, which is higher than in previous studies. This indicates that there is a possibility of VR and Fast Fourier Transform(FFT) processing would affect the accuracy of the model, which means that it is possible to classify a user's emotions toward VR based 3D design alternatives by measuring the EEG with this model.

Keywords

Acknowledgement

Supported by : 한국연구재단

이 논문은 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임.과제번호:NRF-2019R1A2C1088896

References

  1. Campbell, D., & Wells, M. (2012). A Critique of Virtual Reality in the Architectural Design Process. Washington.edu, Retrieved November 20, 2019. from www.hitl.washington.edu/publications/r-94-3/.
  2. Chang, S., Dong, W., & Jun, H. (2018). An EEG-based Deep Neural Network Classification Model for Recognizing Emotion of Users in Early Phase of Design. Journal of the Architectural Institute of Korea, Planning and Design Section, 34(12), 85-94.
  3. Chang, S., & Jun, H. (2019). Hybrid Deep-Learning Model to recognize emotional responses of users towards architectural design alternatives. Journal of Asian Architecture and Building Engineering, 1-11. https://doi.org/10.1080/13467581.2019.1660663.
  4. Chiarelli, A., Croce, P., Meria, A., & Zappasodi, F. (2018). Deep learning for hybrid EEG-fNIRS brain- computer interface: application to motor imagery classification. Journal of Neural Engineering, 15(3), 1-12.
  5. Choi, B. (2019). Diagnosis and classification of Alzheimer's patients using Convolution Neural Network. Thesis, Inje University, 1-2.
  6. Hajinoroozi, M., Mao, Z., & Huang, Y. (2015). Prediction of driver's drowsy and alert states from EEG signals with deep learning. 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 493-496.
  7. Im, C. (2014). Neural Interface and Neroarchitecture. Review of Architecture and Building Science 58(9), 48-52.
  8. Jun, J., & Jo, G. (2016). Deep learning, reading and expressing human emotion. Tech M. Retrieved November 20, 2019 from http://techm.kr/bbs/board.php?bo_table=article&wr_id=2182.
  9. Jung, D., Yoo, S., & Jang, Y. (2019). Motion Sickness Measurement and Analysis in Virtual Reality using Deep Neural Networks Algorithm. Korean Computer Graphics Society, 25(1), 23-32. https://doi.org/10.15701/kcgs.2019.25.1.23
  10. Kang, S. (2016). Network of A.I. & super business. Seoul, Ridibooks, 73-76.
  11. Kim, D., Kim, M., Kim, S., Park, Y., Park, J., Bae, K., Lee, S., Lee, J., Im, C., Jeon, Y., Jin, S., Chae, J., & Hwang, H. (2017). Understanding and Application of EEG. Seoul, Hakjisa, 78-135.
  12. Kim, S. (2016). Deep Learning First Steps: From Machine Learning to Convolutional Neural Networks. Seoul, HanbitMedia, 17-161.
  13. Kim, S., Kang, S., Ji, S., & Jun, H. (2019). Measurement of EEG and Analysis of Stress Change in Space Using Virtual Reality -Focus on the Hitlers residence-. Journal of the Architectural Institute of Korea, Planning and Design Section, 35(8), 73-79.
  14. Kim, Y. (2018). What changes and innovations will come into our personalized lives during the Fourth Industrial Revolution? goesan times, Retrieved November 20, 2019 from http://www.goesantimes.com/news/articleView.html?idxno=10682.
  15. Kumar, S., Shama, A., Mamun, K., & Tsunoda, T. (2016). A Deep Learning Approach for Motor Imagery EEG Signal Classification. 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE), 34-39.
  16. Lee, K. (1998). Human Sesibility and Emotion in Sensibility Ergonomics. Korean Journal of The Science of Emotion & Sensibility, 1(1), 113-122.
  17. Lee, J. (2004). Virtual Reality and Psychology. Korean Journal of Psychology: General, 23(2), 87-104.
  18. Lin, Y., Wang, C., Jung, T., Wu, T., Jeng, S, Buann, J., & Chen, J. (2010). Eeg-based emotion recognition in music listening. IEEE Transactions on Biomedical Engineering, 57(7), 1798-1806. https://doi.org/10.1109/TBME.2010.2048568
  19. Liu, Y., Jiang, X., Cao, T., Wan, F., & Mak, P. (2012). Implementation of SSVEP based BCI with Emotiv EPOC. IEEE International Conference on VECIMS Tianjin, 34-37.
  20. Myung, J., & Jun, H. (2019). Emotional Responses toward 3D Space based on Virtual Reality - Focus on EEG Response to Single-Person Housing with Different Plan Configuration. Journal of the Architectural Institute of Korea, Planning and Design Section, 35(12), 55-64.
  21. Nafjan, A., Hosny, M., & Wabil, A. (2017). Classification of human emotions from electroencephalogram (EEG) signal using deep neural network. International Journal of Advanced Computer Science and Applications(IJACSA), 8(9), 419-425.
  22. Nie, D., Wang, X., Shi, L., & Lu, B. (2011). Eeg-based emotion recognition during watching movies. In 2011 5th IEEE/EMBS international conference on Neural Engineering, 667-670.
  23. Othman, M., Wahab, A., Karim, I., Dzulkifili, M., & Alshaikli, I. (2013). Eeg emotion recognition based on the dimensional models of emotions. Procedia-Social and Behavioral Sciences, 97, 30-37. https://doi.org/10.1016/j.sbspro.2013.10.201
  24. Park, M. (2013). All about brain science. Seoul, Humanist, 588-596.
  25. Park, J. (2009). Marketing meets brain science - Neuromarketing. Excellence Marketing for customer, 43(12), 15-22.
  26. Petrantonakis, P., & Hadjileontiadis, L. (2010). Emotion recognition from eeg using higher order crossings. IEEE Transaction on Information Technology in Biomedicine, 14(2), 186-197. https://doi.org/10.1109/TITB.2009.2034649
  27. Prateek, J. (2017). Artificial Intelligence with Python. Acorn techinal book.
  28. Shemesh, A., Talmon, R., Karp, O., Amir, I., Bar, M., & Grobman, Y. (2016). Affective response to architecture - investigating human reaction to spaces with different geometry. Architectural Science Review, 60(2), 116-125.
  29. Strongman, K. (1996). The Psychology of Emotion (4th ed.). Wiley.
  30. Wang, D., & Linda N. (2013). Architectural Research Methods 2nd Edition. New Jersey, John Wiley & Sons, 351.