Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

A Viewer Preference Model Based on Physiological Feedback

CogTV를 위한 생체신호기반 시청자 선호도 모델

  • 박태서 (서울대학교 인지과학협동과정) ;
  • 김병희 (서울대학교 컴퓨터공학부) ;
  • 장병탁 (서울대학교 인지과학협동과정)
  • Received : 2014.03.09
  • Accepted : 2014.05.08
  • Published : 2014.06.25
Download PDF
⟨ Previous Next ⟩

Abstract

A movie recommendation system is proposed to learn a preference model of a viewer by using multimodal features of a video content and their evoked implicit responses of the viewer in synchronized manner. In this system, facial expression, body posture, and physiological signals are measured to estimate the affective states of the viewer, in accordance with the stimuli consisting of low-level and affective features from video, audio, and text streams. Experimental results show that it is possible to predict arousal response, which is measured by electrodermal activity, of a viewer from auditory and text features in a video stimuli, for estimating interestingness on the video.

본 논문은 TV를 이용한 영화시청 환경에서 해당 컨텐트에 대한 시청자의 암묵적 반응과 컨텐트의 멀티모달 피쳐를 실시간으로 측정 및 동기화하여 이를 기반으로 동영상 선호모델을 지속적으로 개선하고 필요시 영화추천을 수행하는 시스템을 제안한다. 제안한 시스템에선 이미지, 소리, 자막 스트림으로부터 실시간 추출되는 저수준 피쳐들과 동기화되어 측정된 얼굴표정, 자세 및 생체신호로부터 해당 동영상이 유발한 시청자의 감정상태를 추정하여 선호모델 학습에 사용한다. 제안한 컨텐트-시청자 연계 추천모델의 일례로서 컨텐트의 오디오 및 자막 정보를 이용하여 시청자의 피부전기활성도로 측정된 arousal반응을 예측할 수 있음을 보인다.

Keywords

References

  1. F. Ricci, L. Rokach, and B. Shapira, Introduction to Recommender Systems Handbook. Springer US, 2011.
  2. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry, "Using collaborative filtering to weave an information tapestry," Communications of the ACM, vol. 35. pp. 61-70, 1992.
  3. Y. Koren, R. Bell, and C. Volinsky, "Matrix factorization techniques for recommender systems," Computer , vol. 42, no. 8, pp. 30-37, 2009.
  4. P. Lops, M. Gemmis, M. De, & G. Semeraro, "Content-based Recommender Systems: State of the Art and Trends," in Recommender Systems Handbook, F. Ricci and L. Rokach, Eds. Springer US, pp. 73-105, 2011.
  5. S. Sural, G. Q. G. Qian, and S. Pramanik, "Segmentation and histogram generation using the HSV color space for image retrieval," Proceedings of International Conference on Image Processing, vol. 2, pp. 589-592, 2002.
  6. M. J. Black, "Combining intensity and motion for incremental segmentation and tracking over long image sequences," Proceedings of European Conference on Computer Vision, pp. 485-493, 1992.
  7. Y. K. Y. Ke, X. T. X. Tang, and F. J. F. Jing, "The design of high-level features for photo quality assessment," Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 419-426, 2006.
  8. O. Le Meur, T. Baccino, and A. Roumy, "Prediction of the interobserver visual congruency (IOVC) and application to image ranking," Proceedings of the 19th ACM International Conference on Multimedia, pp. 373-382, 2011.
  9. P. Valdez and A. Mehrabian, "Effects of color on emotions," Journal of Experimental Psychology General, vol. 123, no. 4, pp. 394-409, Dec. 1994. https://doi.org/10.1037/0096-3445.123.4.394
  10. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: audio, visual, and spontaneous expressions.," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39-58, Jan. 2009. https://doi.org/10.1109/TPAMI.2008.52
  11. N. Liu, E. Dellandréa, B. Tellez, and L. Chen, "Associating textual features with visual ones to improve affective image classification," Proceedings of the fourth International Conference on Affective Computing and Intelligent Interaction, pp. 195-204, 2011.
  12. Y. Baveye, J.-N. Bettinelli, E. Dellandrea, L. Chen, and C. Chamaret, "A Large Video Data Base for Computational Models of Induced Emotion," Proceedings of the sixth International Conference on Affective Computing and Intelligent Interaction, pp. 13-18, 2013.
  13. P. Mermelstein, "Distance measures for speech recognition, psychological and instrumental," Pattern Recognit. Artif. Intell., vol. 116, pp. 374-388, 1976.
  14. K. R. Scherer, "Vocal affect expression: a review and a model for future research.," Psychol. Bull., vol. 99, pp. 143-165, 1986. https://doi.org/10.1037/0033-2909.99.2.143
  15. D. Neiberg and K. Laskowski, "Emotion Recognition in Spontaneous Speech Using GMMs," Proceedings of INTERSPEECH - ICSLP Ninth International Conference on Spoken Language Processing, pp. 809-812, 2006.
  16. R. Calvo and S. D'Mello, "Affect detection: An interdisciplinary review of models, methods, and their applications," IEEE Trans. Affect. Comput., vol. 1, no. 1, pp. 18-37, 2010. https://doi.org/10.1109/T-AFFC.2010.1
  17. C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak, and S. Hellmann, "DBpedia - A crystallization point for the Web of Data," J. Web Semant., vol. 7, pp. 154-165, 2009. https://doi.org/10.1016/j.websem.2009.07.002
  18. S. Baccianella, A. Esuli, and F. Sebastiani, "SENTIWORDNET 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining," Proceedings of the Seventh International Conference on Language Resources and Evaluation, pp. 2200-2204, 2008.
  19. P. Ekman and W. V Friesen, "Constants across cultures in the face and emotion," J. Pers. Soc. Psychol., vol. 17, pp. 124-129, 1971. https://doi.org/10.1037/h0030377
  20. P. Ekman, "Basic Emotions", in Dalgleish, T; Power, M, Handbook of Cognition and Emotion, Sussex, UK: John Wiley & Sons, 1999.
  21. L. A. Feldman, "Valence-focus and arousal-focus: Individual differences in the structure of affective experience," Journal of Personality and Social Psychology, vol. 69, pp. 153-166, 1995. https://doi.org/10.1037/0022-3514.69.1.153
  22. L. F. Barrett, "Discrete emotions or dimensions? the role of valence focus and arousal focus," Cogn. Emot., vol. 12, pp. 579-599, 1998. https://doi.org/10.1080/026999398379574
  23. D. McDuff, A. Karlson, A. Kapoor, and M. Czerwinski, "AffectAura: an intelligent system for emotional memory," Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 849-858, 2012.
  24. M. E. Dawson and A. M. Schell, "The Electrodermal System," in J. T. Cacioppo and L. G. Tassinary (Eds.), Principles of Psychophysiology: Physical,social, and inferential elements, The Cambridge Press, Cambridge, 1990.
  25. The Hidden Markov Model Toolkit (HTK), Available: http://htk.eng.cam.ac.uk/, 2009, [Accessed: March 3, 2014]
  26. S. Bird, E. Loper and E. Klein, Natural Language Processing with Python. O'Reilly Media Inc., 2009.
  27. G. H. John, P. Langley, "Estimating continuous distributions in Bayesian classifiers," Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338-345, 1995.
  28. D. Aha, D. Kibler, "Instance-based learning algorithms," Machine Learning. vol. 6, pp. 37-66, 1991.
  29. S. le Cessie, J. C. van Houwelingen, "Ridge estimators in logistic regression," Applied Statistics, vol. 41, no. 1, pp. 191-201, 1992. https://doi.org/10.2307/2347628
  30. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, K. R. K. Murthy, "Improvements to Platt's SMO algorithm for SVM classifier design," Neural Computation. vol. 13, no. 3, pp. 637-649, 2001. https://doi.org/10.1162/089976601300014493
  31. R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers, San Mateo, CA, 1993.
  32. L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001. https://doi.org/10.1023/A:1010933404324
  33. K. A. Spackman, "Signal detection theory: Valuable tools for evaluating inductive learning," Proceedings of the Sixth International Workshop on Machine Learning. pp. 160-163, 1989.
  34. S. S. Shapiro, M. B. Wilk, "An analysis of variance test for normality (complete samples)," Biometrika vol. 52, no. 3-4, pp. 591-611, 1965. https://doi.org/10.1093/biomet/52.3-4.591