Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Affective interaction to emotion expressive VR agents

가상현실 에이전트와의 감성적 상호작용 기법

  • 최아영 (가천대학교 소프트웨어학과)
  • Received : 2016.10.28
  • Accepted : 2016.11.30
  • Published : 2016.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

This study evaluate user feedback such as physiological response and facial expression when subjects play a social decision making game with interactive virtual agent partners. In the social decision making game, subjects will invest some of money or credit in one of projects. Their partners (virtual agents) will also invest in one of the projects. They will interact with different kinds of virtual agents which behave reciprocated or unreciprocated behavior while expressing socially affective facial expression. The total money or credit which the subject earns is contingent on partner's choice. From this study, I observed that subject's appraisal of interaction with cooperative/uncooperative (or friendly/unfriendly) virtual agents in an investment game result in increased autonomic and somatic response, and that these responses were observed by physiological signal and facial expression in real time. For assessing user feedback, Photoplethysmography (PPG) sensor, Galvanic skin response (GSR) sensor while capturing front facial image of the subject from web camera were used. After all trials, subjects asked to answer to questions associated with evaluation how much these interaction with virtual agents affect to their appraisals.

최근 인간과 에이전트 간의 감성적인 상호작용을 지원하기 위해 에이전트의 기쁨, 슬픔 등의 얼굴 표정에 대한 사용자의 반응을 분석하는 연구들이 진행되었다. 그러나 기존의 연구에서는 사용자가 긍정적 혹은 부정적으로 평가하는 지를 검증하기 위해 주로 설문지를 통해 사용자 피드백을 획득하는 방법을 사용하였다. 본 연구에서는 인간과 에이전트 상호작용에서 에이전트의 얼굴 표정에 대한 사용자의 생체신호를 측정하여 감성적인 영향이 있는지 살펴보았다. 일관성 있는 생체 신호 해석 결과를 도출하기 위해 실시간으로 심박수와 피부전도도 등의 생체 신호를 획득하고, 명시적 혹은 비명시적인 피드백 분석방법을 통합하여, 해석을 안정적으로 하도록 하였다. 실험결과 에이전트의 긍정(혹은 부정)적인 얼굴 표정이 사용자의 긍정(혹은 부정)적인 반응을 이끌어 냄을 확인할 수 있었다. 또한, 인간과 에이전트와의 상호작용에서도 인간과 인간의 상호작용과 유사하게 에이전트의 긍정적인 얼굴 표정이 사용자의 부정적인 감정을 감소시킴을 확인할 수 있었다. 본 연구는 인간과 에이전트의 감성적인 상호작용을 위해 에이전트의 얼굴 표정을 디자인하기 위한 근거로 활용할 수 있다.

Keywords

Acknowledgement

Supported by : 정보통신기술진흥센터, 한국연구재단

References

  1. J. C. Lester, S. A. Converse, S. E. Kahler, S. T. Barlow, B. A. Stone, and R. S. Bhogal, "The persona effect: Affective impact of animated pedagogical agents," In Proceedings of CHI, pp. 359-366, 1997.
  2. S. W. McQuiggan, J. L. Robison, R. Phillips, J. C. Lester, "Modeling parallel and reactive empathy in virtual agents: an inductive approach," In Proceedings of AAMAS, pp. 167-174, 2008.
  3. H. C. van Vugt, J. F. Hoorn, E. A. Konijn, "Interactive engagement with embodied agents: an empirically validated framework," In Proceedings of CASA, pp.195-204, 2009.
  4. A. Camurri, A. Coglio, "An architecture for emotional agents," IEEE Multimedia, pp. 24-33, 1998.
  5. J. N. Bailenson, E.D. Pontikakis, I.B. Mauss, J.J Gross, M.E. Jabon, C.A. Hutcherson, C. Nass, O. John, "Real-time classification of evoked emotions using facial feature tracking and physiological responses," International Journal of Human Machine Studies, Vol. 66, pp. 303-317, 2008. https://doi.org/10.1016/j.ijhcs.2007.10.011
  6. H. Prendinger, S. Mayer, J. Mori, and M. Ishizuka. "Persona effect revisited: Using bio-signals to measure and reflect the impact of character-based interfaces," In Proceedings of the 4th International Working Conference on Intelligent Virtual Agents, pp. 283-291, 2003.
  7. C. Melo, L. Zheng, J. Gratch, "Expression of Moral Emotions in Cooperating Agents," In Proceedings of International Conference on Intelligent Virtual Agents, 2009.
  8. J. N. Bailenson, E. Aharoni, A. C. Guadagno, R. E. Guadagno, A. Dimov, J. Blascovich, "Comparing behavioral and selfreport measures of embodied agents' social presence in immersive virtual environments," In Proceedings of the 7th Annual International Workshop on presence, 2004.
  9. J. N. Bailenson, K. Swinth, C. Hoyt, S. Persky, A. Dimov, J. Blascovich, "The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments," Presence, Vol. 14, No.4, pp.379-393, 2005. https://doi.org/10.1162/105474605774785235
  10. M. Rodrigo, R. Baker, J. Agapito, J. Nabos, M. Repalam, S. Reyes, M. Pedro, "The Effects of an Interactive Software Agent on Student Affective Dynamics while Using an Intelligent Tutoring System," IEEE Transactions on Affective Computing, Vol.3, No.2, pp.224-236, 2012. https://doi.org/10.1109/T-AFFC.2011.41
  11. M.M. Bradley, M. Codispoti, B.N. Cuthbert, P.J. Lang, "Emotion and motivation I: Defensive and appetitive reactions in picture processing," Emotion, Vol. 1, pp.276-298, 2001. https://doi.org/10.1037/1528-3542.1.3.276
  12. P.J. Lang, "A bio-informational theory of emotional imagery," Psychophysiology, Vol. 16, No. 6, pp. 495-512, 1979. https://doi.org/10.1111/j.1469-8986.1979.tb01511.x
  13. J. K. Rilling, D. R. Goldsmith, A. L. Glenn AL, M. R. Jairam, H. A. Elfenbein, J. E. Dagenais, C. D. Murdock, G. Pagnoni, "The neural correlates of the affective response to unreciprocated cooperation," Neuropsychologia, Vol. 46, No. 5, pp.1256-66, 2008. https://doi.org/10.1016/j.neuropsychologia.2007.11.033
  14. S. Kiesler, L. Sproull, K. Waters, "A Prisoner's dilemma experiment on cooperation with people and human like computers," Journal of Personality and Social Psychology, Vol. 70, No. 1, pp. 47-65, 1996. https://doi.org/10.1037/0022-3514.70.1.47
  15. J. Scharlemann, C. Eckel, A. Kacelnik, R. Wilson, "The value of smile: Game theory with a human face," Journal of Economic Psychology, Vol. 22, No. 5, pp. 617-640, 2001. https://doi.org/10.1016/S0167-4870(01)00059-9
  16. S. Kopelman, A. Rosette, L. Thompson, "The three faces of Eve: Strategic displays of positive, negative and neutral emotions in negotiations," Organizational Behavior and Human Decision Processes, Vol.99, No.1, pp.81-101, 2006. https://doi.org/10.1016/j.obhdp.2005.08.003
  17. W. Yoshida, B. Seymour, K. Friston, and R. Dolan, "Neural Mechanisms of Belief Inference during Cooperative Games," Journal of Neuroscience, Vol. 30, No.32, pp.10744-10751, 2010. https://doi.org/10.1523/JNEUROSCI.5895-09.2010
  18. D. Watson, L. A. Clark, A. Tellegen, A, "Development and validation of brief measures of positive and negative affect: The PANAS scale," Journal of Personality and Social Psychology, Vol. 54, pp.1063-1070, 1988. https://doi.org/10.1037/0022-3514.54.6.1063
  19. T. W. Buchanan, "Retrieval of Emotional Memories," Psychological Bulletin, Vol. 133, No.5, pp.764-779, 2008.
  20. J. Cacioppo, G. Berntson, J. Larsen, K. Poehlmann, T. Ito, "The psychophysiology of emotion," M. Lewis, J. Haviland- Jones, (Eds.), Handbook of emotions, 2nd ed. The Guilford Press: New York, NY, pp. 173-191, 2000.
  21. The impact of positive, negative and neutral stimuli in a virtual reality cognitive-motor rehabilitation task: a pilot study with stroke patients
  22. H. Prendinger, J. Mori, M. Ishizuka, "Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game," International Journal of Human-Computer Studies, Vol. 62, No. 2, pp.231-245, 2005 https://doi.org/10.1016/j.ijhcs.2004.11.009
  23. M. S. Cameirao, A. L. Faria, T. Paulino, J. Alves, and S. B. Bermudez, "The impact of positive, negative and neutral stimuli in a virtual reality cognitive-motor rehabilitation task: a pilot study with stroke patients," Journal of NeuroEngineering and Rehabilitation, Vol. 13, No. 1, pp.1-15, 2016. https://doi.org/10.1186/s12984-015-0109-2