The Journal of the Korea Contents Association (한국콘텐츠학회논문지)

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

The Congruent Effects of Gesture and Facial Expression of Virtual Character on Emotional Perception: What Facial Expression is Significant?

가상 캐릭터의 몸짓과 얼굴표정의 일치가 감성지각에 미치는 영향: 어떤 얼굴표정이 중요한가?

  • 류지헌 (전남대학교 사범대학 교육학과) ;
  • 유승범 (전남대학교 사범대학 교육학과)
  • Received : 2016.01.12
  • Accepted : 2016.02.03
  • Published : 2016.05.28
Download PDF
⟨ Previous Next ⟩

Abstract

In the design and develop a virtual character, it is important to correctly deliver target emotions generated by the combination of facial expression and gesture. The purpose of this study is to examine the effect of congruent/incongruent between gesture and facial expression on target emotion. In this study four emotions, joy, sadness, fear, and anger, are applied. The results of study showed that sadness emotion were incorrectly perceived. Moreover, it was perceived as anger instead of sadness. Sadness can be easily confused when facial expression and gestures were simultaneously presented. However, in the other emotional status, the intended emotional expressions were correctly perceived. The overall evaluation of virtual character's emotional expression was significantly low when joy gesture was combined with sad facial expression. The results of this study suggested that emotional gesture is more influential correctly to deliver target emotions to users. This study suggested that social cues like gender or age of virtual character should be further studied.

디지털 콘텐츠에서 구현되는 가상 캐릭터를 효과적으로 개발하기 위해서는 감성 상태(기쁨, 슬픔, 공포, 화남)가 제대로 전달되도록 하는 것이 중요하다. 이 연구에서는 가상 캐릭터의 얼굴표정과 몸짓이 표현하는 감성 상태의 일치여부에 따라서 사용자가 가상 캐릭터의 감성 상태를 어떻게 지각하며, 가상 캐릭터의 몸짓을 어떻게 평가하는가를 검증하였다. 이를 위하여 몸짓과 얼굴표정의 감성표현을 동일하게 구성한 일치조건, 정반대의 감성표현으로 구성된 불일치조건, 그리고 무표정한 얼굴표정으로 구현된 통제집단을 구성했다. 연구결과에 따르면 슬픔 조건에서 의도된 감성 상태가 제대로 전달되지 않았을 뿐만 아니라 오히려 화난 것으로 지각되었다. 그러나 나머지 감성 상태에서는 몸짓과 얼굴표정의 일치여부는 의도된 감성 전달에 영향을 미치지 않았다. 감성 상태의 표현에 대한 가상 캐릭터의 전체적인 몸짓에 대한 평가에서는 기쁨을 표현한 몸짓이 불일치 조건의 얼굴표정을 갖게 될 때, 평가 점수가 낮았다. 가상 캐릭터의 감성을 표현할 때는 얼굴 표정도 중요하지만 몸짓 자체의 감성표현이 전반적으로 결정적인 역할을 수행하고 있음을 확인할 수 있었다. 가상 캐릭터의 성별이나 연령과 같은 사회적 단서에 대한 연구의 필요성을 설명했다.

Keywords

References

  1. W. L. Johnson, J. W. Rickel, and J. C. Lester, "Animated pedagogical agents: Face-to-face interaction in interactive learning environments," International Journal of Artificial Intelligence in Education, Vol.41, pp.41-78, 2000.
  2. H. van der Meij, J. van der Meij, and R. Harmsen, "Animated pedagogical agents effects on enhancing student motivation and learning in a science inquiry learning environment," Educational Technology Research and Development, Vol.63, No.3, pp.381-403, 2015. https://doi.org/10.1007/s11423-015-9378-5
  3. S. Domagk, "Do Pedagogical Agents Facilitate Learner Motivation and Learning Outcomes?," J. of Media Psychology: Theories, Methods, and Applications, Vol.22, No.2, pp.84-97, 2010. https://doi.org/10.1027/1864-1105/a000011
  4. S. H. Chen and S. Y. Chen, "When educational agents meet surrogate competition: Impacts of competitive educational agents on students' motivation and performance," Computers & Education, Vol.75, pp.274-281, 2014. https://doi.org/10.1016/j.compedu.2014.02.014
  5. 김진옥, "가상 인간의 감정 표현 인식을 위한 비언어적 다중모달 영향 분석," 인터넷정보학회지, Vol.13, No.5, pp.9-19, 2012.
  6. F. D. Schonbrodt and J. B. Asendorpf, "The challenge of constructing psychologically believable agents," J. of Media Psychology: Theories, Methods, and Applications, Vol.23, No.2, pp.100-107, 2011. https://doi.org/10.1027/1864-1105/a000040
  7. Dael, N..Mortillaro and K. R. M..Scherer, "Emotion expression in body action and posture," Emotion, Vol.12, No.5, pp.1085-1101, 2012. https://doi.org/10.1037/a0025737
  8. J. K. Hietanen and J. M. Leppanen, "Judgment of other people's facial expressions of emotions is influenced by their concurrent affective hand movements," J. of Psychology, Vol.49, No.3, pp.221-230, 2008.
  9. M. Courgeon, C. Clavel, N. Tan, and J. C. Martin, "Front View vs. Side View of Facial and Postural Expressions of Emotions in a Virtual Character," Springer Berlin Heidelberg, In Z. Pan, A. Cheok, & W. Muller (Eds.), Transactions on Edutainment VI, Vol.6758, pp.132-143, 2011.
  10. C. Clavel, J. Plessier, J. C. Martin, L. Ach, and B. Morel, "Combining Facial and Postural Expressions of Emotions in a Virtual Character," Springer Berlin Heidelberg, In Z. Ruttkay, M. Kipp, A. Nijholt, & H. Vilhjalmsson (Eds.), Intelligent Virtual Agents, Vol.5773, pp.287-300, 2009.
  11. H. K. Meeren, C. C. van Heijnsbergen, and B. de Gelder, "Rapid perceptual integration of facial expression and emotional body language," Proceedings of the National Academy of Sciences of the United States of America, Vol.102, No.45, pp.16518-16523, 2005.
  12. M. A. H. Rot, K. Hogenelst, and C. M. Gesing, "Communal and agentic behaviour in response to facial emotion expressions," British Journal of Psychology, Vol.105, No.2, pp.173-186, 2014. https://doi.org/10.1111/bjop.12029
  13. E. Gilboa-Schechtman and I. Shachar-Lavie, "More than a face: a unified theoretical perspective on nonverbal social cue processing in social anxiety," Frontiers in Human Neuroscience, Vol.7, No.904, pp.1-13, 2013.
  14. R. Reisenzein, M. Studtmann, and G. Horstmann, "Coherence between Emotion and Facial Expression: Evidence from Laboratory Experiments," Emotion Review, Vol.5, No.1, pp.16-23, 2013. https://doi.org/10.1177/1754073912457228
  15. M. Kipp and J. C. Martin, Gesture and Emotion: Can basic gestural form features discriminate emotions?, Affective Computing and Intelligent Interaction and Workshops, Amsterdam, 2009.
  16. R. Plutchik, Emotions and Life Perspectives from Psychology, Biology, and Evolution, Translated by Park. K, Seoul: Hakjisa, 2004. (Original work published 2002).
  17. S. Sumpeno, M. Hariadi, and M. H. Purnomo, "Facial emotional expressions of life-like character based on text classifier and fuzzy logic," IAENG International Journal of Computer Science, Vol.38, No.2, pp.122-133, 2011.
  18. J. L. Tracy and R. W. Robins, "The nonverbal expression of pride: evidence for cross-cultural recognition," J. of personality and social psychology, Vol.94, No.3, pp.516-530, 2008. https://doi.org/10.1037/0022-3514.94.3.516
  19. 이노미, "문화 간 비언어적 커뮤니케이션에서의 한국과 아시아 지역 손짓언어의 비교 연구," 커뮤니케이션학 연구, Vol.16, No.2, pp.5-34, 2008.
  20. A. Bailey and S. Kelly, "Picture power: Gender versus body language in perceived status," J. of Nonverbal Behavior, Vol.39, No.4, pp.317-337, 2015. https://doi.org/10.1007/s10919-015-0212-x
  21. A. S. Won, L. Yu, J. H. Janssen, and J. N. Bailenson, "Tracking Gesture to Detect Gender," International Society for Presence Research Annual Conference, Philadelphia, PA, USA, 2012.