Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Motion based Autonomous Emotion Recognition System: A Preliminary Study on Bodily Map according to Type of Emotional Stimuli

동작 기반 Autonomous Emotion Recognition 시스템: 감정 유도 자극에 따른 신체 맵 형성을 중심으로

  • Jungeun Bae (Department of Computer and Software, Hanyang University) ;
  • Myeongul Jung (Department of Computer and Software, Hanyang University) ;
  • Youngwug Cho (Department of Computer and Software, Hanyang University) ;
  • Hyungsook Kim (Graduate School of Public Policy, Hanyang University) ;
  • Kwanguk (Kenny) Kim (Department of Computer and Software, Hanyang University)
  • 배정은 (한양대학교 컴퓨터.소프트웨어학과) ;
  • 정면걸 (한양대학교 컴퓨터.소프트웨어학과) ;
  • 조영욱 (한양대학교 컴퓨터.소프트웨어학과) ;
  • 김형숙 (한양대학교 공공정책대학원) ;
  • 김광욱 (한양대학교 컴퓨터.소프트웨어학과)
  • Received : 2023.06.12
  • Accepted : 2023.07.05
  • Published : 2023.07.25
Download PDF
⟨ Previous Next ⟩

Abstract

Not only emotions affect physical sensations, but they also have an impact on physical movements. The responses to emotions vary depending on the type of emotional stimuli. However, research on the effects of emotional stimuli on the activation of bodily movements has not been rigorously examined, and these effects have not been investigated in Autonomous Emotion Recognition (AER) systems. In this study, we aimed to compare the emotional responses of 20 participants to three types of emotional stimuli (words, pictures, and videos) and investigate their activation or deactivation for the AER system. Our dependent measures included emotional responses, computer-based self-reporting methods, and bodily movements recorded using motion capture devices. The results suggested that video stimuli elicited higher levels of emotional movement, and emotional movement patterns were similar across different types of emotional stimuli for happiness, sadness, anger, and neutrality. Additionally, the findings indicated that bodily changes observed during video stimuli had the highest classification accuracy. These findings have implications for future research on the bodily changes elicited by emotional stimuli.

기존 연구에 따르면 감정은 신체 감각 및 신체 움직임과 같은 신체적 변화에 영향을 주고, 감정 자극에 따라 다르게 나타난다고 알려져 있다. 그러나, 감정의 자극에 따른 신체 감각 및 신체 움직임의 활성화 정도 및 Autonomous emotion recognition(AER) 시스템의 성능에 미치는 영향에 대한 연구는 아직 알려져 있지 않다. 본 연구에서는 20명의 피험자를 대상으로 3가지 종류의 감정 자극(단어, 사진, 영상)을 활용하여 AER 시스템에 미치는 영향을 연구하였다. 측정 변인으로는 정서적 반응, 컴퓨터 기반 자가 보고, Motion Capture 장비를 통해 측정한 신체 움직임을 활용하였다. 본 연구의 결과를 통하여 영상 자극이 다른 자극에 비해 더 많은 신체 움직임을 유도하는 것을 확인하고, 영상 자극을 통해 수집한 신체적 특이점이 AER을 위한 분류 정확도 역시 가장 높음을 확인하였다. 신체 움직임을 기반으로 한 감정적 특이점은 행복, 놀람, 분노, 중립 등에서 감정 유도 자극의 종류에 따라 비슷한 패턴이 나타남을 확인하였다. 본 연구의 결과는 향후 신체적 변화를 기반으로 한 AER 시스템 연구에 기여할 수 있을 것으로 기대된다.

Keywords

Acknowledgement

이 논문은 2021 년도 정부(과기정통부)의 재원으로 한국연구재단 바이오·의료기술개발사업의 지원을 받아 수행된 연구임(No. NRF-2021M3A9E4080780). 이 논문은 2018 년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No.2018R1A5A7059549).

References

  1. Cacioppo J. and Gardner W., "Emotion", Annual Review of Psychology, pp. 191-214, 1999. 
  2. Yang, H., Ciftci, U. and Yin L., "Facial expression recognition by de-expression residue learning", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2168-2177, 2018. 
  3. Shan. L. and Weihong. D., "Deep Facial Expression Recognition: A Survey", IEEE Transactions on Affective Computing, pp. 1195-1215, 2022. 
  4. Gavrilescu M., "Recognizing emotions from videos by studying facial expressions, body postures and hand gestures," 2015 23rd Telecommunications Forum Telfor (TELFOR), pp. 720-723, 2015. 
  5. Nourhan E., Pablo B., German I. and Stefan W., "Emotion Recognition from Body Expressions with a Neural Network Architecture", In Proceedings of the 5th International Conference on Human Agent Interaction (HAI '17), pp. 143-149, 2017. 
  6. Cho. Y., Jung M. and Kim K., "Emotion and Body Movement: A Comparative Study of Automatic Emotion Recognition Using Body Motions," 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 768-771, 2022. 
  7. Nummenmaa, L., Glerean, E., Hari, R., and Hietanen, J., "Bodily maps of emotions", Proceedings of the National Academy of Sciences, pp. 646-651, 2014. 
  8. Wallbott H., "Bodily expression of emotion", European Journal of Social Psychology, pp. 879-896, 1998. 
  9. Stephanie O., Christoph M., Micah M., Christine M., Serge C. and Theodor L., "Electrical neuroimaging reveals early generator modulation to emotional words", NeuroImage, pp. 1242-1251, 2004. 
  10. Anne K., Delphine G. and Nicolas V., "Congruent bodily arousal promotes the constructive recognition of emotional words", Consciousness and Cognition, pp. 81-88, 2017. 
  11. Lang P., Bradley M. and Cuthbert B., The International Affective Picture System (IAPS): Technical manual and Affective Ratings", Emotion and Attention, 1997 
  12. Bradley M. and Lang P., "The International Affective Picture System (IAPS) in the study of emotion and attention", Handbook of emotion elicitation and assessment, pp. 29-46, 2007. 
  13. Mikels, J., Fredrickson B., Larkin G., Lindberg C., Maglio S., and Reuter-Lorenz P., "Emotional category data on images from the International Affective Picture System", Behavior Research Methods, pp. 626-630, 2005. 
  14. Tina M., Lench H., Flores S. and Bench S., "Discrete emotions predict changes in cognition, judgment, experience, behavior, and physiology: A meta-analysis of experimental emotion elicitations", Psychological Bulletin, pp. 834-855, 2011. 
  15. Siedlecka E. and Denson, T. "Experimental Methods for Inducing Basic Emotions: A Qualitative Review", Emotion Review, pp. 87-97, 2019. 
  16. Gross J. and Levenson R., "Emotion elicitation using films", Cognition & Emotion, pp. 87-108, 1995. 
  17. Schaefer A., Nils F., Sanchez X. and Philippot P., "Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers", Cognition and Emotion, pp. 1153-1172, 2010. 
  18. Bradley M. and Lang P., "Measuring emotion: The self-assessment manikin and the semantic differential", Journal of Behavior Therapy and Experimental Psychiatry, pp. 49-59, 1994. 
  19. Shun L., Liqing C., Changye Z., Baobin L., Nan Z. and Tingshao Z., "Emotion recognition using Kinect motion capture data of human gaits", PeerJ, 2016. 
  20. Ferdous A., Hossain B. and Marina L., "Emotion Recognition From Body Movement", IEEE Access, pp. 11761-11781, 2020 
  21. Pollick F., Paterson H., Bruderlin A. and Sanford A. "Perceiving affect from arm movement", Cognition. pp. 51-61, 2001. 
  22. Saha S., Datta S., Konar A. and Janarthanan R., "A study on emotion recognition from body gestures using Kinect sensor, "International Conference on Communication and Signal Processing, pp. 56-60, 2014. 
  23. Lundqvist D., Flykt A. and Ohman A., "The Karolinska Directed Emotional Faces: KDEF CD-ROM (Karolinska Institutet, Stockholm)", Psychology, 1998. 
  24. Livingstone S. and Russo F., "The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English", PLoS ONE , pp. 2018. 
  25. Mayer J., Allen I. and Beauregard K. "Mood inductions for four specific moods: A procedure employing guided imagery", Journal of Mental Imagery, pp. 133-150, 1995. 
  26. Mosher D. and White B. "Effects of committed or casual erotic guided imagery on females' subjective sexual arousal and emotional response.", Journal of Sex Research, pp. 273-299, 1980. 
  27. Clark, D., "On the induction of depressed mood in the laboratory: Evaluation and comparison of the Velten and musical procedures", Advances in Behaviour Research and Therapy, pp. 27-49, 1983. 
  28. Vastfjall, D., "Emotion induction through music: A review of the musical mood induction procedure.", Musicae Scientiae, pp. 173-211, 2002. 
  29. Ed G., "Laban Movement Analysis: Charting the Ineffable Domain of human Movement", Journal of Physical Education, Recreation & Dance, pp. 27-30, 1995.