KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

General Relation Extraction Using Probabilistic Crossover

확률적 교차 연산을 이용한 보편적 관계 추출

  • 이제승 (한국해양대학교 컴퓨터공학과) ;
  • 김재훈 (한국해양대학교 컴퓨터공학과 및 해양인공지능융합전공)
  • Received : 2023.05.23
  • Accepted : 2023.07.19
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Relation extraction is to extract relationships between named entities from text. Traditionally, relation extraction methods only extract relations between predetermined subject and object entities. However, in end-to-end relation extraction, all possible relations must be extracted by considering the positions of the subject and object for each pair of entities, and so this method uses time and resources inefficiently. To alleviate this problem, this paper proposes a method that sets directions based on the positions of the subject and object, and extracts relations according to the directions. The proposed method utilizes existing relation extraction data to generate direction labels indicating the direction in which the subject points to the object in the sentence, adds entity position tokens and entity type to sentences to predict the directions using a pre-trained language model (KLUE-RoBERTa-base, RoBERTa-base), and generates representations of subject and object entities through probabilistic crossover operation. Then, we make use of these representations to extract relations. Experimental results show that the proposed model performs about 3 ~ 4%p better than a method for predicting integrated labels. In addition, when learning Korean and English data using the proposed model, the performance was 1.7%p higher in English than in Korean due to the number of data and language disorder and the values of the parameters that produce the best performance were different. By excluding the number of directional cases, the proposed model can reduce the waste of resources in end-to-end relation extraction.

관계 추출은 텍스트로부터 개체(named entity) 사이의 관계를 추출하는 과정이다. 전통적으로 관계 추출 방법은 주어와 목적어가 미리 정해진 상태에서 관계만 추출한다. 그러나 종단형 관계 추출에서는 개체 쌍마다 주어와 목적어의 위치를 고려하여 가능한 모든 관계를 추출해야 하므로 이 방법은 시간과 자원을 비효율적으로 사용한다. 본 논문에서는 이러한 문제를 완화하기 위해 문장에서 주어와 목적어의 위치에 따른 방향을 설정하고, 정해진 방향에 따라 관계를 추출하는 방법을 제안한다. 제안하는 방법은 기존의 관계 추출 데이터를 활용하여 문장에서 주어가 목적어를 가리키는 방향을 나타내는 방향 표지를 새롭게 생성하고, 개체 위치 토큰과 개체 유형 정보를 문장에 추가하는 작업을 통해 사전학습 언어모델 (KLUE-RoBERTa-base, RoBERTa-base)을 이용하여 방향을 예측한다. 그리고 확률적 교차 연산을 통해 주어와 목적어 개체의 표상을 생성한다. 이후 이러한 개체의 표상을 활용하여 관계를 추출한다. 실험 결과를 통해, 제안 모델이 하나로 통합된 라벨을 예측하는 것보다 3 ~ 4%p 정도 더 우수한 성능을 보여주었다. 또한, 제안 모델을 이용해 한국어 데이터와 영어 데이터를 학습할 때, 데이터 수와 언어적 차이로 인해 한국어보다 영어에서 1.7%p 정도 더 높은 성능을 보여주었고, 최상의 성능을 내는 매개변수의 값이 다르게 나타나는 부분도 관찰할 수 있었다. 제안 모델은 방향에 따른 경우의 수를 제외함으로써 종단형 관계 추출에서 자원의 낭비를 줄일 수 있다.

Keywords

References

  1. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed representations of words and phrases and their compositionality," Proceedings of the 26th International Conference on Neural Information Processing Systems, Vol.2, pp.3111-3119, 2013.
  2. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching word vectors with subword information," Transactions of the Association for Computational Linguistics, Vol.5, pp.135-146, 2017. https://doi.org/10.1162/tacl_a_00051
  3. J. Devlin, M. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol.1, pp.4171-4186, 2019.
  4. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and Veselin Stoyanov, RoBERTa: A Robustly Optimized BERT Pretraining Approach, arXiv:1907.11692, 2019.
  5. K. Clark, M. Luong, Q. V. Le, and C. D. Manning, "ELECTRA: Pre-training text encoders as discriminators rather than generators," Proceedings of International Conference on Learning Representations, 2019.
  6. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, 2020, "ALBERT: A lite BERT for self-supervised learning of language representations," Proceedings of International Conference on Learning Representations.
  7. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, "BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension," Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.7871-7880, 2020.
  8. H. Wang, M. Tan, M. Yu, S. Chang, D. Wang, K. Xu, X. Guo, and S. Potdar, "Extracting multiple-relations in onepass with pre-trained transformers," Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.1371-1377, 2019.
  9. L. B. Soares, N. Fitzgerald, J. Ling, and T. Kwiatkowski, "Matching the blanks: distributional similarity for relation learning," Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.2895-2905, 2019.
  10. Y. Hur, S. Son, M. Shim, J. Lim, and H. Lim, "K-EPIC: Entity-perceived context representation in Korean relation extraction," Applied Sciences, Vol.11, No.23, pp.11472, 2021.
  11. W. Zhou and M. Chen, "An improved baseline for sentence-level relation extraction," arXiv:2102.01373, 2021.
  12. S. Lyu and H. Chen, "Relation classification with entity type restriction," Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp.390-395, 2021.
  13. J. Lee and J. Kim, "Korean relation extraction using pretrained language model and GCN," Proceedings of the 34th Annual Conference on Human and Cognitive Language Technology, pp.379-384, 2022.
  14. Z. Zhong and D. Chen, "A frustratingly easy approach for entity and relation extraction," Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.50-61, 2021.
  15. D. Ye, Y. Lin, P. Li, and M. Sun, "Packed levitated marker for entity and relation extraction," Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Vol.1, pp.4904-4917, 2022.
  16. S. Park et al., KLUE: Korean Language Understanding Evaluation, arXiv:2105.09680, 2021.
  17. Y. Zhang, V. Zhong, D. Chen, G. Angeli, and C. D. Manning, "Position aware attention and supervised data improve slot filling," In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp.35-45, 2017.
  18. Y. Cui, M. Jia, T. Lin, Y. Song, and S. Belongie, "Class-balanced loss based on effective number of samples," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.9268-9277, 2019.