Healthcare Informatics Research

Korean Society of Medical Infomatics (대한의료정보학회)
권호 표지
DOI QR코드

DOI QR Code

Arousal and Valence Classification Model Based on Long Short-Term Memory and DEAP Data for Mental Healthcare Management

  • Choi, Eun Jeong (Department of Computer Science, Graduate School, Sangmyung University) ;
  • Kim, Dong Keun (Department of Intelligent Engineering Informatics for Human, College of Convergence Engineering, Sangmyung University)
  • Received : 2018.08.21
  • Accepted : 2018.10.22
  • Published : 2018.10.31
⟨ Previous Next ⟩

Abstract

Objectives: Both the valence and arousal components of affect are important considerations when managing mental healthcare because they are associated with affective and physiological responses. Research on arousal and valence analysis, which uses images, texts, and physiological signals that employ deep learning, is actively underway; research investigating how to improve the recognition rate is needed. The goal of this research was to design a deep learning framework and model to classify arousal and valence, indicating positive and negative degrees of emotion as high or low. Methods: The proposed arousal and valence classification model to analyze the affective state was tested using data from 40 channels provided by a dataset for emotion analysis using electrocardiography (EEG), physiological, and video signals (the DEAP dataset). Experiments were based on 10 selected featured central and peripheral nervous system data points, using long short-term memory (LSTM) as a deep learning method. Results: The arousal and valence were classified and visualized on a two-dimensional coordinate plane. Profiles were designed depending on the number of hidden layers, nodes, and hyperparameters according to the error rate. The experimental results show an arousal and valence classification model accuracy of 74.65 and 78%, respectively. The proposed model performed better than previous other models. Conclusions: The proposed model appears to be effective in analyzing arousal and valence; specifically, it is expected that affective analysis using physiological signals based on LSTM will be possible without manual feature extraction. In a future study, the classification model will be adopted in mental healthcare management systems.

Keywords

References

  1. Salamon N, Grimm JM, Horack JM, Newton EK. Application of virtual reality for crew mental health in extended-duration space missions. Acta Astronaut 2018;146:117-22. https://doi.org/10.1016/j.actaastro.2018.02.034
  2. Valmaggia LR, Latif L, Kempton MJ, Rus-Calafell M. Virtual reality in the psychological treatment for mental health problems: an systematic review of recent evidence. Psychiatry Res 2016;236:189-95. https://doi.org/10.1016/j.psychres.2016.01.015
  3. Faust O, Hagiwara Y, Hong TJ, Lih OS, Acharya UR. Deep learning for healthcare applications based on physiological signals: a review. Comput Methods Programs Biomed 2018;161:1-13. https://doi.org/10.1016/j.cmpb.2018.04.005
  4. Zhang Q, Chen X, Zhan Q, Yang T, Xia A. Respirationbased emotion recognition with deep learning. Comput Ind 2017;92-93:84-90. https://doi.org/10.1016/j.compind.2017.04.005
  5. Yin Z, Zhao M, Wang Y, Yang J, Zhang J. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Programs Biomed 2017;140:93-110. https://doi.org/10.1016/j.cmpb.2016.12.005
  6. Haeyen S, van Hooren S, van der Veld WM, Hutschemaekers G. Promoting mental health versus reducing mental illness in art therapy with patients with personality disorders: a quantitative study. Arts Psychother 2018;58:11-6. https://doi.org/10.1016/j.aip.2017.12.009
  7. Zhang L, Kong M, Li Z. Emotion regulation difficulties and moral judgement in different domains: the mediation of emotional valence and arousal. Pers Individ Dif 2017;109:56-60. https://doi.org/10.1016/j.paid.2016.12.049
  8. Healey J, Picard RW. Detecting stress during real-world driving tasks using physiological sensors. IEEE trans Intell Transp Syst 2005;6(2):156-66. https://doi.org/10.1109/TITS.2005.848368
  9. Xia L, Malik AS, Subhani AR. A physiological signalbased method for early mental-stress detection. Biomed Signal Process Control 2018;46:18-32. https://doi.org/10.1016/j.bspc.2018.06.004
  10. Lee D, Kim JH, Jung WH, Lee HJ, Lee SG. The study on EEG based emotion recognition using the EMD and FFT. Proceedings of the HCI Society of Korea; 2013 Jan 30-Feb 1; Jeongseon, Korea. p. 127-30.
  11. Wang D, Shang Y. Modeling physiological data with deep belief networks. Int J Inf Educ Technol 2013;3(5):505-11.
  12. DEAPdataset: a dataset for emotion analysis using EEG, physiological and video signal [Internet]. London: Queen Mary University of London; c2017 [cited at 2018 Oct 1]. Available from: http://www.eecs.qmul.ac.uk/mmv/datasets/deap.
  13. Kim J, Andre E. Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 2008;30(12):2067-83. https://doi.org/10.1109/TPAMI.2008.26
  14. Kreibig SD. Autonomic nervous system activity in emotion: a review. Biol Psychol 2010;84(3):394-421. https://doi.org/10.1016/j.biopsycho.2010.03.010
  15. Collet C, Vernet-Maury E, Delhomme G, Dittmar A. Autonomic nervous system response patterns specificity to basic emotions. J Auton Nerv Syst 1997;62(1-2):45-57. https://doi.org/10.1016/S0165-1838(96)00108-7
  16. Ryoo DW, Kim YS, Lee JW. Wearable systems for service based on physiological signals. Conf Proc IEEE Eng Med Biol Soc 2005;3:2437-40.
  17. Jerritta S, Murugappan M, Nagarajan R, Wan K. Physiological signals based human emotion recognition: a review. Proceedings of 2011 IEEE 7th International Colloquium on Signal Processing and its Applications (CSPA); 2001 Mar 4-6; Penang, Malaysia. p. 410-5.
  18. Chang CY, Zheng JY, Wang CJ. Based on support vector regression for emotion recognition using physiological signals. Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN); 2010 Jul 18-23; Barcelona, Spain. p. 1-7.
  19. Kim JH, Whang MC, Kim YJ, Woo JC. The study on emotion recognition by time dependent parameters of autonomic nervous response. Korean J Sci Emot Sensib 2008;11:637-44.
  20. Vogt T, Andre E. Improving automatic emotion recognition from speech via gender differentiation. Proceedings of Language Resources and Evaluation Conference (LREC); 2006 May 24-26; Genoa, Italy.
  21. Mill A, Allik J, Realo A, Valk R. Age-related differences in emotion recognition ability: a cross-sectional study. Emotion 2009;9(5):619-30. https://doi.org/10.1037/a0016562
  22. Russell JA. A circumplex model of affect. J Pers Soc Psychol 1980;39(6):1161-78. https://doi.org/10.1037/h0077714
  23. Kwon SJ. Sentiment analysis of movie reviews using the Word2vec and RNN [Master's thesis]. Seoul, Korea: Dongguk University; 2009.
  24. TensorFlow. Recurrent neural network [Internet]. [place unknown]: TensorFlow; c2018 [cited at 2018 Oct 1]. Available from: https://www.tensorflow.org/tutorials/sequences/recurrent.
  25. Chanel G, Kierkels JJ, Soleymani M, Pun T. Short-term emotion assessment in a recall paradigm. Int J Hum Comput Stud 2009;67(8):607-27. https://doi.org/10.1016/j.ijhcs.2009.03.005
  26. Rainville P, Bechara A, Naqvi N, Damasio AR. Basic emotions are associated with distinct patterns of cardiorespiratory activity. Int J Psychophysiol 2006;61(1):5-18. https://doi.org/10.1016/j.ijpsycho.2005.10.024
  27. Deeplearning4j [Internet]. Ottawa, Canada: Eclipse Foundation; c2018 [cited at 2018 Oct 1]. Available from https://deeplearning4j.org/about.
  28. Machine learning: learning rate, data processing, overfitting [Internet]. [place unknown: publisher unknown]; c2017 [cited at 2018 Oct 1]. Available from http://copycode.tistory.com/166.
  29. Song SH. The emotion analysis based on long short term memory using the central and autonomic nervous system signals [Master's thesis]. Seoul, Korea: Sangmyung University; 2018.

Cited by

  1. Multi-method Fusion of Cross-Subject Emotion Recognition Based on High-Dimensional EEG Features vol.13, pp.None, 2019, https://doi.org/10.3389/fncom.2019.00053
  2. Deep Learning in Physiological Signal Data: A Survey vol.20, pp.4, 2020, https://doi.org/10.3390/s20040969
  3. Machine-Learning-Based Emotion Recognition System Using EEG Signals vol.9, pp.4, 2018, https://doi.org/10.3390/computers9040095
  4. Application of Electroencephalography-Based Machine Learning in Emotion Recognition: A Review vol.15, pp.None, 2018, https://doi.org/10.3389/fnsys.2021.729707
  5. Impact of Visual Design Elements and Principles in Human Electroencephalogram Brain Activity Assessed with Spectral Methods and Convolutional Neural Networks vol.21, pp.14, 2018, https://doi.org/10.3390/s21144695