IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Korean Head-Tail Tokenization and Part-of-Speech Tagging by using Deep Learning

딥러닝을 이용한 한국어 Head-Tail 토큰화 기법과 품사 태깅

  • Received : 2022.07.07
  • Accepted : 2022.08.08
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Korean is an agglutinative language, and one or more morphemes are combined to form a single word. Part-of-speech tagging method separates each morpheme from a word and attaches a part-of-speech tag. In this study, we propose a new Korean part-of-speech tagging method based on the Head-Tail tokenization technique that divides a word into a lexical morpheme part and a grammatical morpheme part without decomposing compound words. In this method, the Head-Tail is divided by the syllable boundary without restoring irregular deformation or abbreviated syllables. Korean part-of-speech tagger was implemented using the Head-Tail tokenization and deep learning technique. In order to solve the problem that a large number of complex tags are generated due to the segmented tags and the tagging accuracy is low, we reduced the number of tags to a complex tag composed of large classification tags, and as a result, we improved the tagging accuracy. The performance of the Head-Tail part-of-speech tagger was experimented by using BERT, syllable bigram, and subword bigram embedding, and both syllable bigram and subword bigram embedding showed improvement in performance compared to general BERT. Part-of-speech tagging was performed by integrating the Head-Tail tokenization model and the simplified part-of-speech tagging model, achieving 98.99% word unit accuracy and 99.08% token unit accuracy. As a result of the experiment, it was found that the performance of part-of-speech tagging improved when the maximum token length was limited to twice the number of words.

Keywords

Acknowledgement

본 논문은 정부 (과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임 (NRF-2021R1F1A1061433).

References

  1. D. H. Park, W. S. Choi, H. J. Kim, S. L. Lee, "Web Document Classification Based on Hangeul Morpheme and Keyword Analyses", Journal of Korean Information Processing Society D, Vol. 19, No. 4, pp. 263-270, 2012 (in Korean).
  2. W. Cho, G. Shin, W. Lee, S. Son, H. Song, J. Lee, H. Lee, S. Jo, "KoELECTRA-Based Named Entity Recognition Using Korean Morphological Analyzers", Proceedings of 2021 Korea Computer Congress, pp. 1897-1899, 2021 (in Korean).
  3. Y. Ha, J. Cheon, I. Wang, M. Park, G. Woo, "A Filtering Method of Malicious Comments Through Morpheme Analysis", Journal of Korea Contents Association, Vol. 21, No. 9, pp. 750-761, 2021 (in Korean). https://doi.org/10.5392/JKCA.2021.21.09.750
  4. S. S. Kang, "Analysis of Korean Irregular Verbs by using Syllable Characteristics", Proceedings of 5th Hangul and Korean Information Processing, pp. 385-394, 1993 (in Korean).
  5. S. S. Kang, "Multi-level Morphology and Morphological Analysis Model for Korean", Proceedings of 6th Hangul and Korean Information Processing, pp. 140-145, 1994 (in Korean).
  6. J. S. Lee, "Three-Step Probabilistic Model for Korean Morphological Analysis", Journal of KIISE, Vol. 38, No. 5, pp. 257-268, 2011 (in Korean).
  7. D. Bahdanau, K. Cho, Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate", ICLR 2015, arXiv:1409.0473, 2015.
  8. A. Vaswan, N. Shazeer., N. Parmar., J. Uszkoreit., L. Jones.,N. Aidan. Gomez, L. Kaiser., L. Polosukhin., "Attention Is All You Need", Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS), arXiv:1706.03762v5, 2017.
  9. J. Devlin., M. W. Chang., K. Lee., K. Toutanova., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", preprint arXiv:1810.04805, 2018.
  10. H. Y. Lee, S. S. Kang, "Bi-LSTM-CRF and Syllable Embedding for Automatic Spacing of Korean Sentences", Proceedings of 30th Hangul and Korean Information Processing, pp. 605-607, 2018 (in Korean).
  11. K. S. Shim, "Automatic Word Spacing based on Conditional Random Fields", Korean Journal of Cognitive Science, Vol. 22, No. 2, pp. 217-233, 2011 (in Korean). https://doi.org/10.19066/COGSCI.2011.22.2.007
  12. H. Y. Lee, S. S. Kang, "Compound Noun Decomposition by using Syllable-based Embedding and Deep Learning", Journal of Smart Media, Vol. 8, No. 2, pp. 74-79, 2019 (in Korean).
  13. J. H. Shin, H. R. Park, "A Statistical Model for Korean Text Segmentation by using Syllable-level Bigrams", Proceedings of 9th Hangul and Korean Information Processing, pp. 255-260, 1997 (in Korean).
  14. S. S. Kang, "Automatic Correction of Word-spacing Errors by using Syllable Bigram", Journal of Speech Science, Vol. 8, No. 2, pp. 83-90, 2001 (in Korean).
  15. D. H. Lim, Y. J. Chun, H. J. Kim, S. S. Kang, "Word Segmentation by using Extended Syllable Bigram", Proceedings of 17th Hangul and Korean Information Processing, pp. 189-193, 2005 (in Korean).
  16. T. Kudo.., "Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 66-75, 2018.
  17. T. Kudo, J. Richardson., "SentencePiece: A Simple and Language Independent Subword Tokenizer nd Detokenizer for Neural Text Processing", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System, pp. 66-71. 2018.
  18. J. Lafferty, A. McCallum, F. C. N. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data", Proceedings of the 18th International Conference on Machine Learning 2001, pp. 282-289, 2001.
  19. T. Brants., "TnT - A Statistical Part-of-Speech Tagger", Proceedings of the 6th Applied Natural Language Processing Conference, pp. 224-231, 2000.
  20. J. Youn, J. Lee, "A Deep Learning-based Two-Steps Pipeline Model for Korean Morphological Analysis and Part-of-Speech Tagging", Journal of KIISE, Vol. 48, No. 4, pp. 444-452, 2021. https://doi.org/10.5626/JOK.2021.48.4.444
  21. J. Li, E. H. Lee, J. H. Lee, "Sequence-to-sequence based Morphological Analysis and Part-Of-Speech Tagging for Korean Language with Convolutional Features", Journal of KIISE, Vol. 44, No. 1, pp. 57-62, 2017 (in Korean). https://doi.org/10.5626/JOK.2017.44.1.57
  22. Y. Choi, K. Lee, "Performance Analysis of Korean Morphological Analyzer based on Transformer and BERT", Journal of KIISE, Vol. 47, No. 8, pp. 730-741, 2020 (in Korean). https://doi.org/10.5626/jok.2020.47.8.730
  23. J. M. Kim, H. J. Seo, S. S. Kang, "Korean Head-Tail POS-Tagger by using Transformer", Proceedings of 33th Hangul and Korean Information Processing, pp. 544-547, 2021 (in Korean).
  24. D. Kim, Y. Bae, C. Ock, H. Choi, C. Kim, "Korean POS and Homonym Tagging System using HMM", Proceedings of 20th Hangul and Korean Information Processing, pp. 12-16, 2008 (in Korean).
  25. J. H. Kim, "Korean Part-of-Speech Tagging using a Weighted Network", Journal of KIISE, Vol. 25, No. 6, pp. 951-959, 1998 (in Korean).
  26. K. S. Shim, "Syllable-based POS Tagging without Korean Morphological Analysis", Korean Journal of Cognitive Science, Vol. 22, No. 3, pp. 327-345, 2011 (in Korean). https://doi.org/10.19066/COGSCI.2011.22.3.005
  27. J. M. Kim, H. J. Seo, S. S. Kang, "Korean Head-Tail POS-Tagger by using Transformer", Journal of Smart Media, Vol. 11, No. 5, pp. 17-25, 2022 (in Korean).
  28. D. Cho, "Hate Speech Detection by using Word Embedding and Deep Learning Model", M.S. Thesis, Kookmin University, 2021 (in Korean).