KIISE Transactions on Computing Practices (정보과학회 컴퓨팅의 실제 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Social Network Analysis of TV Drama via Location Knowledge-learned Deep Hypernetworks

장소 정보를 학습한 딥하이퍼넷 기반 TV드라마 소셜 네트워크 분석

  • 남장군 (서울대학교 컴퓨터공학부) ;
  • 김경민 (서울대학교 컴퓨터공학부) ;
  • 장병탁 (서울대학교 컴퓨터공학부)
  • Received : 2016.03.16
  • Accepted : 2016.08.23
  • Published : 2016.11.15
⟨ Previous Next ⟩

Abstract

Social-aware video displays not only the relationships between characters but also diverse information on topics such as economics, politics and culture as a story unfolds. Particularly, the speaking habits and behavioral patterns of people in different situations are very important for the analysis of social relationships. However, when dealing with this dynamic multi-modal data, it is difficult for a computer to analyze the drama data effectively. To solve this problem, previous studies employed the deep concept hierarchy (DCH) model to automatically construct and analyze social networks in a TV drama. Nevertheless, since location knowledge was not included, they can only analyze the social network as a whole in stories. In this research, we include location knowledge and analyze the social relations in different locations. We adopt data from approximately 4400 minutes of a TV drama Friends as our dataset. We process face recognition on the characters by using a convolutional- recursive neural networks model and utilize a bag of features model to classify scenes. Then, in different scenes, we establish the social network between the characters by using a deep concept hierarchy model and analyze the change in the social network while the stories unfold.

Social-aware video는 자유로운 스토리 전개를 통해 인물들간의 관계뿐만 아니라 경제, 정치, 문화 등 다양한 지식을 사람에게 전달해주고 있다. 특히 장소에 따른 사람들간의 대화 습성과 행동 패턴은 사회관계를 분석하는데 있어서 아주 중요한 정보이다. 하지만 멀티모달과 동적인 특성으로 인해 컴퓨터가 비디오로부터 자동으로 지식을 습득하기에는 아직 많은 어려움이 있다. 이러한 문제점들을 해결하기 위해 기존의 연구에서는 딥하이퍼넷 모델을 사용하여 드라마 등장인물의 시각과 언어 정보를 기반으로 계층적 구조를 사용해 소셜 네트워크를 분석하였다. 하지만 장소 정보를 사용하지 않아 전반적인 스토리로부터 소셜 네트워크를 분석할 수밖에 없었다. 본 논문에서는 기존 연구를 바탕으로 장소 정보를 추가하여 각 장소에서의 인물 특성을 분석해 보았다. 본 논문에서는 총 4400분 분량의 TV드라마 "Friends"를 사용했고 C-RNN모델을 통해 등장인물을 인식하였으며 Bag of Features로 장소를 분류하였다. 그리고 딥하이퍼넷 모델을 통해 자동으로 소셜 네트워크를 생성하였고 각 장소에서의 인물 관계 변화를 분석하였다.

Keywords

Acknowledgement

Supported by : 정보통신기술진흥센터, 한국산업기술평가관리원, 국방과학연구소

References

  1. V. Ramanathan, B. Yao, and L. Fei-Fei, "Social Role Discovery in Human Events," Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR2013), pp. 2475-2482, 2013.
  2. C.-Y. Weng, W.-T. Chu, and J.-L. Wu, "RoleNet: Movie Analysis from the Perspective of Social Networks," Proc. of IEEE Transactions on Multimedia (TMM2009), pp. 256-271, 2009.
  3. C.-J. Nan, K. M. Kim, and B.-T. Zhang, "Social Network Analysis of TV Drama Characters via Deep Concept Hierarchies," Proc. of International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2015), pp. 831-836, 2015.
  4. J.-W. Ha, K.-M. Kim, and B.-T. Zhang, "Automated Construction of Visual-linguistic Knowledge via Concept Learning from Cartoon Videos," Proc. of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015), pp. 522-528, 2015.
  5. B.-T. Zhang, J.-W. Ha, and M. Kang, "Sparse Population Code Models of Word Learning in Concept Drift," Proc. of Annual Meeting of the Cognitive Science Society (CogSci 2012), pp. 1221-1226, 2012.
  6. R. Socher, B. Huval, B. Bath, C. D. Manning, and A. Y. Ng, "Convolutional-recursive Deep Learning for 3D Object Classification," Proc. of International Conference on Advances in Neural Information Processing Systems(NIPS2012), pp. 665-673, 2012.
  7. L. Fei-Fei, "A Bayesian Hierarchical Model for Learning Natural Scene Categories," Proc. of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR2005), Vol. 2, pp. 524-531, 2005.
  8. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," Proc. of International Conference on Advances in Neural Information Processing Systems (NIPS 2012), pp. 1097-1105, 2012.
  9. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed Representation of Words and Phrases and Their Compositionality," Proc. of International Conference on Advances in Neural Information Processing Systems (NIPS 2013), pp. 3111-3119, 2013.