The Journal of Society for e-Business Studies (한국전자거래학회지)

Society for e-Business Studies (한국전자거래학회)
권호 표지
DOI QR코드

DOI QR Code

An Intelligent Chatbot Utilizing BERT Model and Knowledge Graph

BERT 모델과 지식 그래프를 활용한 지능형 챗봇

  • Yoo, SoYeop (Department of Software, Gachon University) ;
  • Jeong, OkRan (Department of Software, Gachon University)
  • Received : 2019.06.28
  • Accepted : 2019.08.09
  • Published : 2019.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

As artificial intelligence is actively studied, it is being applied to various fields such as image, video and natural language processing. The natural language processing, in particular, is being studied to enable computers to understand the languages spoken and spoken by people and is considered one of the most important areas in artificial intelligence technology. In natural language processing, it is a complex, but important to make computers learn to understand a person's common sense and generate results based on the person's common sense. Knowledge graphs, which are linked using the relationship of words, have the advantage of being able to learn common sense easily from computers. However, the existing knowledge graphs are organized only by focusing on specific languages and fields and have limitations that cannot respond to neologisms. In this paper, we propose an intelligent chatbotsystem that collects and analyzed data in real time to build an automatically scalable knowledge graph and utilizes it as the base data. In particular, the fine-tuned BERT-based for relation extraction is to be applied to auto-growing graph to improve performance. And, we have developed a chatbot that can learn human common sense using auto-growing knowledge graph, it verifies the availability and performance of the knowledge graph.

인공지능이 활발하게 연구되면서 이미지, 영상, 자연어 처리와 같은 다양한 분야에 적용되고 있다. 특히 자연어 처리 분야는 사람이 말하고 쓰는 언어들을 컴퓨터가 이해할 수 있도록 하기 위한 연구들이 진행되고 있고 인공지능 기술에서 매우 중요한 영역 중 하나로 여겨진다. 자연어 처리에서 컴퓨터에게 사람의 상식을 이해할 수 있도록 학습시키고 사람의 상식을 기반으로 결과를 생성하도록 하는 것은 복잡하지만 중요한 기술이다. 단어들의 관계를 이용해 연결한 지식 그래프는 컴퓨터에게 쉽게 상식을 학습시킬 수 있다는 장점이 있다. 하지만 기존에 고안된 지식 그래프들은 특정 언어나 분야에만 집중해 구성되어 있거나 신조어 등에는 대응하지 못하는 한계점을 갖고 있다. 본 논문에서는 실시간으로 데이터를 수집 및 분석하여 자동으로 확장 가능한 지식 그래프를 구축하고, 이를 기반 데이터로 활용하는 챗봇 시스템을 제안하고자 한다. 특히 자동 확장 그래프에 BERT 기반의 관계 추출 모델을 적용시켜 성능을 향상시키고자 한다. 자동 확장 지식 그래프를 이용해 상식이 학습되어 있는 챗봇을 구축하여 지식 그래프의 활용 가능성과 성능을 검증한다.

Keywords

References

  1. Alsubaiee, S., Altowim, Y., Altwaijry, H., Behm, A., Borkar, V., Bu, Y., and Gabrielova, E., “AsterixDB: A scalable, open source BDMS,” Proceedings of the VLDB Endowment, Vol. 7, No. 14, pp. 1905-1916, 2014. https://doi.org/10.14778/2733085.2733096
  2. Athreya, R. G., Ngonga Ngomo, A. C., and Usbeck, R., "Enhancing Community Interactions with Data-Driven Chatbots-The DBpedia Chatbot," In Companion of the The Web Conference 2018, pp. 143-146, 2018.
  3. Devlin, J., Chang, M. W., Lee, K., and Toutanova, K., "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810. 04805, 2018.
  4. Fellbaum, C., "WordNet: An Electronic Lexical Database," Cambridge, MA: MIT Press, 1998.
  5. Hyun, Y. J. and Kim, N. G., “Text Miningbased Fake News Detection Using News And Social Media Data,” The Journal of Society for e-Business Studies, Vol. 23, No. 4, pp. 19-39, 2018. https://doi.org/10.7838/JSEBS.2018.23.4.019
  6. Ji, G., He, S., Xu, L., Liu, K., and Zhao, J., "Knowledge graph embedding via dynamic mapping matrix," In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Vol. 1, pp. 687-696, 2015.
  7. Lee, D. H. and Kim, K. H., “Web Site Keyword Selection Method by Considering Semantic Similarity Based on Word2Vec,” The Journal of Society for e-Business Studies, Vol. 23, No. 2, pp. 83-96, 2018. https://doi.org/10.7838/JSEBS.2018.23.2.083
  8. Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X., "Learning entity and relation embeddings for knowledge graph completion," In Twenty-ninth AAAI conference on artificial intelligence, 2015.
  9. Mahdisoltani, F., Biega, J., Suchanek, F. M., "YAGO3: A Knowledge Base from Multilingual Wikipedias," Conference on Innovative Data Systems Research (CIDR), 2015.
  10. Paulheim, H., “Knowledge graph refinement: A survey of approaches and evaluation method,” Semantic web, Vol. 8, No. 3, pp. 489-508, 2017. https://doi.org/10.3233/SW-160218
  11. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L., "Deep contextualized word representations," arXiv preprint arXiv:1802. 05365, 2018.
  12. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I., "Language models are unsupervised multitask learners," OpenAI Blog, Vol. 1, No. 8, 2019.
  13. Speer, R., Chin, J., and Havasi, C., "Conceptnet 5.5: An open multilingual graph of general knowledge," In Thirty-First AAAI Conference on Artificial Intelligence, Feb. 2017.
  14. Tarau, P. and Figa, E., "Knowledge-based conversational agents and virtual storytelling," In Proceedings of the 2004 ACM symposium on Applied computing, pp. 39-44, 2004.
  15. Wu, W., Li, H., Wang, H., and Zhu, K. Q., "Probase: A probabilistic taxonomy for text understanding," In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 481-492, 2012.
  16. Yang, Y. J., Lee, B. H., Kim, J. S., and Lee, K. Y., “Development of An Automatic Classification System for Game Reviews Based on Word Embedding,” The Journal of Society for e-Business Studies, Vol. 24, No. 2, pp. 1-14, 2019.
  17. Yoo, S., Song, J., and Jeong, O., "Social media contents based sentiment analysis and prediction system," Expert Systems with Applications, Vol. 105, pp. 102-111, 2018. https://doi.org/10.1016/j.eswa.2018.03.055
  18. Zhang, Y., Peng, Q., and Christopher D. M., "Graph Convolution over Pruned Dependency Trees Improves Relation Extraction," In Proceeding of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2205-2215, 2018.
  19. Zhang, Y., Zhong, V., Chen, D., Angeli, G., and Manning, C. D., "Position-aware attention and supervised data improve slot filling," In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 35-45, 2017.