KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Korean Dependency Parsing Using Stack-Pointer Networks and Subtree Information

스택-포인터 네트워크와 부분 트리 정보를 이용한 한국어 의존 구문 분석

  • 최용석 (충남대학교 전자전파정보통신공학과) ;
  • 이공주 (충남대학교 전파정보통신공학과)
  • Received : 2021.01.05
  • Accepted : 2021.02.04
  • Published : 2021.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this work, we develop a Korean dependency parser based on a stack-pointer network that consists of a pointer network and an internal stack. The parser has an encoder and decoder and builds a dependency tree for an input sentence in a depth-first manner. The encoder of the parser encodes an input sentence, and the decoder selects a child for the word at the top of the stack at each step. Since the parser has the internal stack where a search path is stored, the parser can utilize information of previously derived subtrees when selecting a child node. Previous studies used only a grandparent and the most recently visited sibling without considering a subtree structure. In this paper, we introduce graph attention networks that can represent a previously derived subtree. Then we modify our parser based on the stack-pointer network to utilize subtree information produced by the graph attention networks. After training the dependency parser using Sejong and Everyone's corpus, we evaluate the parser's performance. Experimental results show that the proposed parser achieves better performance than the previous approaches at sentence-level accuracies when adopting 2-depth graph attention networks.

본 연구에서는 포인터 네트워크 모델을 의존 구문 분석에 맞게 확장한 스택-포인터 네트워크 모델을 이용하여 한국어 의존 구문 분석기를 구현한다. 스택-포인터 네트워크 모델 기반 의존 구문 분석기는 인코더-디코더로 구성되어 있으며 다른 의존 구문 분석기와 달리 내부 스택을 갖고 있어 루트부터 시작하는 하향식 구문 분석이 가능하다. 디코더의 각 단계에서는 의존소를 찾기 위해 부모 노드뿐만 아니라 이미 파생된 트리 구조에서 조부모와 형제 노드를 참조할 수 있다. 기존 연구에서는 단순하게 해당 노드들의 합을 계산하여 입력으로 사용하였고, 형제 노드의 경우에는 가장 최근에 방문했던 것만을 사용할 수 있었다. 본 연구에서는 그래프 어텐션 네트워크를 도입하여 이미 파생된 부분 트리를 표현하고 이를 스택-포인터 네트워크의 입력으로 사용하도록 구문 분석기를 수정한다. 세종 코퍼스와 모두의 코퍼스를 대상을 실험한 결과 레이어 2의 그래프 어텐션 네트워크를 이용하여 부분 트리를 표현했을 때 특히 문장 단위의 구문 분석 정확도에서 많은 성능 향상을 확인할 수 있었다.

Keywords

Acknowledgement

이 논문은 2019년 대한민국 교육부와 한국연구재단의 지원을 받아 수행된 연구임(NRF-2019S1A5A2A03041296).

References

  1. S.-H. Na, J. Li, J.-H. Shin, and K. Kim, "Stack LSTMs with Recurrent Controllers for Korean Dependency Parsing," in Proceedings of the 43th Annual Conference on KIISE, pp.446-448, 2016.
  2. J.-W. Min and S.-H. Na, "SyntaxNet Models using Transition Based Recurrent Unit for Korean Dependency Parsing," in Proceedings of the KIISE Korea Computer Congress, pp.602-604, 2017.
  3. S.-H. Na, J. Li, J.-H. Shin, and K. Kim, "Deep Biaffine Attention for Korean Dependency Parsing," in Proceedings of the KIISE Korea Computer Congress, pp.584-586, 2017.
  4. C. Park, H. Hwang, C. Lee, and H. Kim, "Korean Dependency Parsing with Multi-layer Pointer Networks," in Proceedings of the 29th Annual Conference on Human and Cognitive Language Technology, pp.92-96, 2017.
  5. Y. Choi and K. J. Lee, "Korean Dependency Parser using Higher-order features and Stack-Pointer Networks," Journal of KIISE, Vol.46, No.7, pp.636-643, 2019. https://doi.org/10.5626/jok.2019.46.7.636
  6. C. Park, C. Lee, J.-H. Lim, and H. Kim, "Korean Dependency Parsing with BERT," in Proceedings of the KIISE Korea Computer Congress, pp.530-532, 2019.
  7. J. Min, S.-H. Na, J.-H. Shin, and Y.-K. Kim, "RoBERTa for Korean Natural Language Processing: Named Entity Recognition, Sentiment Analysis, Dependency Parsing," in Proceedings of the KIISE Korea Software Congress, pp.407-409, 2019.
  8. J.-W. Min, S.-Y. Hong, Y.-H. Lee, and S.-H. Na, "Graph Neural Networks for Korean Dependency Parsing," in Proceedings of the 31th Annual Conference on Human and Cognitive Language Technology, pp.537-539, 2019.
  9. J.-H. Lim, Y. Bae, H. Kim, Y. Kim, and K.-C. Lee, "Korean Dependency Guidelines for Dependency Parsing and ExoBrain Language Analysis Corpus," in Proceedings of the 27th Annual Conference on Human and Cognitive Language Technology, pp.234-239, 2015.
  10. T. Dozat and C.D. Manning, "Deep biaffine attention for neural dependency parsing," arXiv preprint arXiv:1611.01734, 2016.
  11. C. Dyer, M. Ballesteros, W. Ling, A. Matthews, and N.A. Smith, "Transition-based dependency parsing with stack long short-term memory," arXiv preprint arXiv:1505.08075, 2015.
  12. X. Ma, Z. Hu, J. Liu, N. Peng, G. Neubig, and E. Hovy, "Stack-pointer networks for dependency parsing," arXiv preprint arXiv:1805.01087, 2018.
  13. O. Vinyals, M. Fortunato, and N. Jaitly, "Pointer networks," in Advances in Neural Information Processing Systems, pp.2692-2700, 2015.
  14. T. Ji, Y. Wu, and M. Lan, "Graph-based dependency parsing with graph neural networks," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.2475-2485, 2019.
  15. P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, "Graph attention networks," arXiv preprint arXiv:1710.10903, 2017.
  16. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
  17. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, "Roberta: A robustly optimized bert pretraining approach," arXiv preprint arXiv:1907.11692, 2019.
  18. Y. Wu, M. Schuster, Z. Chen, Q.V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, and K. Macherey, "Google's neural machine translation system: Bridging the gap between human and machine translation," arXiv preprint arXiv:1609.08144, 2016.
  19. CORPUS, "21st Century Sejong Project". 2010: The National Institute of the Korean Language.
  20. C. Lee, J. Kim, and J. Kim, "Korean Dependency Parsing using Deep Learning," in Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology, pp.87-91, 2014.
  21. CORPUS. National Institute of the Korean Language Morphological Analysis Corpus (Version 1.0). 2020, [Internet] https://corpus.korean.go.kr/.
  22. CORPUS. National Institute of the Korean Language Parsing corpus (Version 1.0). 2020, [Internet] https://corpus.korean.go.kr/.