Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Knowledge Embedding Method for Implementing a Generative Question-Answering Chat System

생성 기반 질의응답 채팅 시스템 구현을 위한 지식 임베딩 방법

  • 김시형 (강원대학교 컴퓨터정보통신공학과) ;
  • 이현구 (강원대학교 컴퓨터정보통신공학과) ;
  • 김학수 (강원대학교 컴퓨터정보통신공학과)
  • Received : 2017.07.27
  • Accepted : 2017.11.21
  • Published : 2018.02.15
⟨ Previous Next ⟩

Abstract

A chat system is a computer program that understands user's miscellaneous utterances and generates appropriate responses. Sometimes a chat system needs to answer users' simple information-seeking questions. However, previous generative chat systems do not consider how to embed knowledge entities (i.e., subjects and objects in triple knowledge), essential elements for question-answering. The previous chat models have a disadvantage that they generate same responses although knowledge entities in users' utterances are changed. To alleviate this problem, we propose a knowledge entity embedding method for improving question-answering accuracies of a generative chat system. The proposed method uses a Siamese recurrent neural network for embedding knowledge entities and their synonyms. For experiments, we implemented a sequence-to-sequence model in which subjects and predicates are encoded and objects are decoded. The proposed embedding method showed 12.48% higher accuracies than the conventional embedding method based on a convolutional neural network.

채팅 시스템은 사람의 말을 기계가 이해하고 적절한 응답을 하는 시스템이다. 채팅 시스템은 사용자의 간단한 정보 검색 질문에 대답해야 하는 경우가 있다. 그러나 기존의 생성 채팅 시스템들은 질의응답에 필요한 정보인 지식 개체(트리플 형태 지식에서의 주어와 목적어)의 임베딩을 고려하지 않아 발화에 나타나는 지식 개체가 다르더라도 같은 형태의 답변이 생성되었다. 본 논문에서는 생성 기반 채팅 시스템의 질의응답 정확도를 향상시키기 위한 지식 임베딩 방법을 제안한다. 개체와 유의어의 지식 임베딩을 위해 샴 순환 신경망을 사용하며 이를 이용해 주어와 술어를 인코딩 하고 목적어를 디코딩하는 sequence-to-sequence 모델의 성능을 향상 시켰다. 자체 구축한 채팅데이터를 통한 실험에서 제안된 임베딩 방법은 종래의 합성곱 신경망을 통한 임베딩 방법 보다 12.48% 높은 정확도를 보였다.

Keywords

Acknowledgement

Grant : 언어학습을 위한 자유발화형 음성대화처리 원천기술 개발

Supported by : 정보통신기술진흥센터, 한국연구재단

References

  1. J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan, "A diversity-promoting objective function for neural conversation models," arXiv preprint arXiv:1510.03055, 2015.
  2. L. Shang, Z. Lu, and H. Li, "Neural responding machine for short-text conversation," arXiv preprint arXiv:1503.02364, 2015.
  3. S. Yang and J. Kim, "A New Morphological Analysis for the Spoken Language Translation System," The Journal of the Acoustical Society of Korea, Vol. 18, No. 4, pp. 17-22, 1999. (in Korean)
  4. S. Kim and H. kim, "Chatting System that Pseudomorpheme-based Korean," Proc. of the HCLT, pp. 263-267, 2016. (in Korean)
  5. A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko, "Translating embeddings for modeling multi-relational data," Proc. of NIPS, pp. 2787-2795, 2013.
  6. T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space," arXiv preprint arXiv:1301.3781, 2013.
  7. Z. Wang, J. Zhang, J. Feng, and Z. Chen, "Knowledge Graph Embedding by Translating on Hyperplanes," Proc. of AAAI, pp. 1112-1119, 2014.
  8. Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu, "Learning Entity and Relation Embeddings for Knowledge Graph Completion," Proc. of NIPS, pp. 2181-2187, 2015.
  9. W. Huang, G. Li, and Z. Jin, "Improved Knowledge Base Completion by Path-Augmented TransR Model," arXiv preprint arXiv:1610.04073, 2016.
  10. Y. Lin, Z. Liu, H. Luan, M. Sun, S. Rao, and S. Liu, "Modeling relation paths for representation learning of knowledge bases," arXiv preprint arXiv:1506.00379, 2015.
  11. J. Berant, A. Chou, R. Frostig, and P. Liang, "Semantic Parsing on Freebase from Question-Answer Pairs," Proc. of EMNLP, Vol. 2, No. 5, pp. 6, Oct. 2013.
  12. X. Yao, and B. Van Durme, "Information Extraction over Structured Data: Question Answering with Freebase," Proc. of ACL, pp. 956-966, 2014.
  13. S. W. T. Yih, X. He, and C. Meek, "Semantic parsing for single-relation question answering," Proc. of ACL, 2014.
  14. X. Yao, "Lean Question Answering over Freebase from Scratch," Proc. of HLT-NAACL, pp. 66-70, May. 2015.
  15. H. Bast, and E. Haussmann, "More accurate question answering on freebase," Proc. of ACM, pp. 1431-1440, Oct. 2015.
  16. D. Golub, and X. He, "Character-level question answering with attention," arXiv preprint arXiv: 1604.00727, 2016.
  17. S. Chopra, R. Hadsell, and Y. LeCun, "Learning a similarity metric discriminatively, with application to face verification," Proc. of IEEE, Vol. 1, pp. 539-546, Jun. 2005.
  18. P. Neculoiu, M. Versteegh, M. Rotaru, and T. B. Amsterdam, "Learning Text Similarity with Siamese Recurrent Networks," Proc. of ACL, 2016.
  19. J. Mueller, and A. Thyagarajan, "Siamese Recurrent Architectures for Learning Sentence Similarity," Proc. of AAAI, pp. 2786-2792, Feb. 2016.
  20. F. J. Pineda, "Generalization of back-propagation to recurrent neural networks," Physical review letters, Vol. 59, No. 19, pp. 2229-2232, 1987. https://doi.org/10.1103/PhysRevLett.59.2229
  21. S. Hochreiter, and J. Schmidhuber, "Long shortterm memory," Journal of Neural computation, Vol. 9, No. 8, pp. 1735-1780, Nov. 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  22. K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk and Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," arXiv preprint arXiv:1406.1078, 2014.
  23. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling," arXiv preprint arXiv:1412.3555, 2014.
  24. Y. LeCun, S. Chopra, R. Hadsell, R. Marc'Aurelio, and F. Huang, "A Tutorial on Energy-Based Learning," Predicting Structured Data MIT Press, 2006.
  25. D. Bahdanau, K. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate," arXiv preprint arXiv:1409.0473, 2014.
  26. S. Wiseman, and A. M. Rush, "Sequence-to-sequence learning as beam-search optimization," arXiv preprint arXiv:1606.02960, 2016.
  27. J. L. Ba, J. R. Kiros, and G. E. Hinton, "Layer normalization," arXiv preprint arXiv:1607.06450, 2016.
  28. DBPedia version 2016-04, Available : http://wiki.dbpedia.org/dbpedia-version-2016-04 (downloaded 2017, May. 10)