Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Mention Detection with Pointer Networks

포인터 네트워크를 이용한 멘션탐지

  • Received : 2016.11.30
  • Accepted : 2017.05.23
  • Published : 2017.08.15
⟨ Previous Next ⟩

Abstract

Mention detection systems use nouns or noun phrases as a head and construct a chunk of text that defines any meaning, including a modifier. The term "mention detection" relates to the extraction of mentions in a document. In the mentions, a coreference resolution pertains to finding out if various mentions have the same meaning to each other. A pointer network is a model based on a recurrent neural network (RNN) encoder-decoder, and outputs a list of elements that correspond to input sequence. In this paper, we propose the use of mention detection using pointer networks. Our proposed model can solve the problem of overlapped mention detection, an issue that could not be solved by sequence labeling when applying the pointer network to the mention detection. As a result of this experiment, performance of the proposed mention detection model showed an F1 of 80.07%, a 7.65%p higher than rule-based mention detection; a co-reference resolution performance using this mention detection model showed a CoNLL F1 of 52.67% (mention boundary), and a CoNLL F1 of 60.11% (head boundary) that is high, 7.68%p, or 1.5%p more than coreference resolution using rule-based mention detection.

멘션(mention)은 명사 또는 명사구를 중심어로 가지며, 수식어를 포함하여 어떤 의미를 정의하는 구(chunk)를 구성한다. 문장 내에서 멘션을 추출하는 것을 멘션탐지라 한다. 멘션들 중에서 서로 같은 의미의 멘션들을 찾아내는 것을 상호참조해결이라 한다. 포인터 네트워크는 RNN encoder-decoder 모델을 기반으로, 주어진 입력 열에 대응되는 위치를 출력 결과로 갖는 모델이다. 본 논문에서는 멘션탐지에 포인터 네트워크를 이용할 것을 제안한다. 멘션탐지에 포인터 네트워크를 적용하면 기존의 순차 문제로는 해결할 수 없었던 중첩된 멘션탐지 문제를 해결할 수 있다. 실험 결과, 본 논문에서 제안한 멘션탐지의 성능이 규칙기반 보다 7.65%p 이상 높은 F1 80.07%를 보였으며, 이를 이용한 상호참조해결 성능이 CoNLL F1 56.67%(멘션 경계), 60.11%(중심어 경계)로 규칙기반 멘션탐지를 이용한 상호참조해결에 비하여 7.68%p, 1.5%p 더 좋은 성능을 보였다.

Keywords

Acknowledgement

Grant : (엑소브레인-1세부) 휴먼 지식증강 서비스를 위한 지능진화형 WiseQA 플랫폼 기술 개발

Supported by : 정보통신기술진흥센터

References

  1. O. Vinyals, M. Fortunato, and N. Jaitly, Pointer Networks, Advances in Neural Information Processing Systems, pp. 2674-2682, 2015.
  2. K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representation using RNN encoder-decoder for statistical machine translation, Proc. of EMNLP' 14, 2014.
  3. D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473, 2014.
  4. C. H. Park, G. H. Choi, and C. Lee, "Korean Coreference Resolution using the Multi-pass Sieve," Journal of KIISE, 41.11 (2014.11): 992-1005. 2014. (in Korean) https://doi.org/10.5626/JOK.2014.41.11.992
  5. C. Park, C. Lee, "Mention Detection using Bidirectional LSTM-CRF Model," Proc. of the KIISE for HCLT (2015), pp. 224-227, 2015. (in Korean)
  6. C. Park, C. Lee, "Mention Detection in the Coreference Resolution using the Deep Learning," Proc. Of the KIISE and the KBS Joint Symposium, 2015.
  7. C. Park, C. Lee, "Coreference Resolution for Pronouns with Pointer Networks," Proc. of the KIISE Korea Computer Congress 2016, pp. 699-701, 2016. (in Korean)
  8. C. Lee, J. Kim, and J. Kim, "Korean Dependency Parsing using Deep Learning," Proc. Of the KIISE for HCLT (2014), pp. 87-91, 2014. (in Korean)
  9. Q. Li, H. Ji, Incremental Joint Extraction of Entity Mentions and Relations, Proc. of ACL' 14, 2014.
  10. Q. Li, H. Ji, Y. Hong, and S. Li, Constructing information networks using one single model, Proc. of EMNLP' 14, 2014.
  11. TH. Nguyen, A. Sil, G. Dinu, and R. Florian, Toward Mention Detection Robustness with Recurrent Neural Networks, arXive preprint arXiv: 1602.07749, 2016.