Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Word Sense Disambiguation Using Bidirectional Recurrent Neural Network for Korean Language

  • Min, Jihong (Dept. of Information and Communication Engineering, Inha University) ;
  • Jeon, Joon-Woo (Dept. of Information and Communication Engineering, Inha University) ;
  • Song, Kwang-Ho (Dept. of Information and Communication Engineering, Inha University) ;
  • Kim, Yoo-Sung (Dept. of Information and Communication Engineering, Inha University)
  • Received : 2017.01.24
  • Accepted : 2017.03.20
  • Published : 2017.04.28
Download PDF
⟨ Previous Next ⟩

Abstract

Word sense disambiguation(WSD) that determines the exact meaning of homonym which can be used in different meanings even in one form is very important to understand the semantical meaning of text document. Many recent researches on WSD have widely used NNLM(Neural Network Language Model) in which neural network is used to represent a document into vectors and to analyze its semantics. Among the previous WSD researches using NNLM, RNN(Recurrent Neural Network) model has better performance than other models because RNN model can reflect the occurrence order of words in addition to the word appearance information in a document. However, since RNN model uses only the forward order of word occurrences in a document, it is not able to reflect natural language's characteristics that later words can affect the meanings of the preceding words. In this paper, we propose a WSD scheme using Bidirectional RNN that can reflect not only the forward order but also the backward order of word occurrences in a document. From the experiments, the accuracy of the proposed model is higher than that of previous method using RNN. Hence, it is confirmed that bidirectional order information of word occurrences is useful for WSD in Korean language.

Keywords

References

  1. M. Sundermeyer, and R. Schluter, "LSTM Neural Networks for Language Modeling" Interspeech, pp. 194-197, Sep. 2012.
  2. mykang, bgkim and jslee, "Word Sense Disambiguation using Word2Vec", Proceeding of 27th Conference on Human & Cognitive Language Technology, pp. 81-84, Oct. 2015
  3. jcshin, and, cyock, "Homograph Word Sense Disambiguation using Korean Lexical Semantic Map(UWordMap) and Word-Embedding", Proceeding of Korean Computer Congress 2016, pp. 702-704, Jun. 2016
  4. D. Yuan, J. Richardson, R. Doherty, C. Evans and E. Altendorf, "Word Sense Disambiguation with Neural Language Models", arXivpreprint arXiv:1603.07012, 2016.
  5. jhmin, jwjeon, khsong, and yskim, "Study on Word Sense Disambiguation Using Recurrent Neural Network for Korean", Proceeding of Winter Conference on Korean Association of Computer Education 2017, pp. 93-96, Jan. 2017.
  6. jsbae, and cklee, "End-to-end Learning of Korean Semantic Role Labeling Using Bidirectional LSTM CRF", Proceeding of 42th Winter Conference on Korean Institute of Information Scientists and Engineers, pp. 566-568, Dec. 2015.
  7. C. Irsoy, and C. Cardie, "Opinion Mining with Deep Recurrent Neural Networks", Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, pp. 720-728, 2014.
  8. H. Sak, A. Senior, and F. Beaufays, " Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition", arXiv preprint arXiv:1402.1128, 2014.
  9. Junyoung Chung, C. Gulcehre, KyungHyun Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling", arXiv preprint arXiv:1412.3555, 2014.
  10. shjung, "Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithms", Journal of The Korea Society of Computer and Information, Vol. 17, No. 3, pp. 11-25, Mar. 2012 https://doi.org/10.9708/jksci.2012.17.3.011
  11. hmkim, jmyoon, jhan, kmbae, and yjko, "Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp. 3-8, Oct. 2016
  12. smhan, "Deep Learning Architectures and Applications", Journal of Intelligence Information System, Vol. 22, No. 2, pp. 127-142, Jun. 2016 https://doi.org/10.13088/jiis.2016.22.2.127
  13. Y. Bengio, P. Simard, and P. Frasconi, "Learning Long-Term Dependencies with Gradient Descent is Difficult", IEEE Transactions on Neural Networks, Vol. 5, No. 2, Mar. 1994
  14. bmkang, "Text Context and Word Meaning: Latent Semantic Analysis", Journal of the Linguistic Society of Korea, Vol. 68, pp. 3-34, Apr. 2014.
  15. Gensim, https://radimrehurek.com/gensim/models/word2vec.html
  16. T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space". arXivpreprint arXiv: 1301.3781. 2013.
  17. mbchung, "Color matching application which can help color blind people based on smart phone", Journal of The Korea Society of Computer and Information, Vol.20, No. 5, pp. 65-72, May. 2015 https://doi.org/10.9708/jksci.2015.20.5.065
  18. National Institute of Korean Language, http://ithub.korean.go.kr,
  19. hgkim, mbkang, and jhhong, "21st Century Sejong Modern Korean Corpora: Results and Expectations", Proceeding of 19th Conference on Human & Cognitive Language Technology, pp. 311-316, Oct. 2007
  20. shchoi, jsseol and sglee, "On Word Embedding Models and Parameters Optimized for Korean", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp. 252-256, Oct. 2016
  21. Korean Wikipedia, https://ko.wikipedia.org/
  22. Namuwiki, https://namu.wiki/
  23. SENSEAVAL-2, http://www.hipposmond.com/senseval2/
  24. Keras, http://keras.io

Cited by

  1. 양방향 LSTM을 적용한 단어의미 중의성 해소 감정분석 vol.5, pp.1, 2020, https://doi.org/10.36498/kbigdt.2020.5.1.197
  2. Improving the Performance of Vietnamese-Korean Neural Machine Translation with Contextual Embedding vol.11, pp.23, 2017, https://doi.org/10.3390/app112311119