Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Symbolizing Numbers to Improve Neural Machine Translation

숫자 기호화를 통한 신경기계번역 성능 향상

  • Kang, Cheongwoong (School of Computer Science and Electrical Engineering, Handong Global University) ;
  • Ro, Youngheon (School of Computer Science and Electrical Engineering, Handong Global University) ;
  • Kim, Jisu (School of Computer Science and Electrical Engineering, Handong Global University) ;
  • Choi, Heeyoul (School of Computer Science and Electrical Engineering, Handong Global University)
  • 강청웅 (한동대학교 전산전자공학부) ;
  • 노영헌 (한동대학교 전산전자공학부) ;
  • 김지수 (한동대학교 전산전자공학부) ;
  • 최희열 (한동대학교 전산전자공학부)
  • Received : 2018.05.20
  • Accepted : 2018.06.25
  • Published : 2018.06.30
⟨ Previous Next ⟩

Abstract

The development of machine learning has enabled machines to perform delicate tasks that only humans could do, and thus many companies have introduced machine learning based translators. Existing translators have good performances but they have problems in number translation. The translators often mistranslate numbers when the input sentence includes a large number. Furthermore, the output sentence structure completely changes even if only one number in the input sentence changes. In this paper, first, we optimized a neural machine translation model architecture that uses bidirectional RNN, LSTM, and the attention mechanism through data cleansing and changing the dictionary size. Then, we implemented a number-processing algorithm specialized in number translation and applied it to the neural machine translation model to solve the problems above. The paper includes the data cleansing method, an optimal dictionary size and the number-processing algorithm, as well as experiment results for translation performance based on the BLEU score.

기계 학습의 발전은 인간만이 할 수 있었던 섬세한 작업들을 기계가 할 수 있도록 이끌었고, 이에 따라 많은 기업체들은 기계 학습 기반의 번역기를 출시하였다. 현재 상용화된 번역기들은 우수한 성능을 보이지만 숫자 번역에서 문제가 발생하는 것을 발견했다. 번역기들은번역할문장에 큰숫자가 있을경우종종숫자를잘못번역하며, 같은문장에서숫자만바꿔번역할 때문장의구조를 완전히바꾸어 번역하기도 한다. 이러한 문제점은오번역의 가능성을 높이기 때문에해결해야 될 사안으로여겨진다. 본 논문에서는 Bidirectional RNN (Recurrent Neural Network), LSTM (Long Short Term Memory networks), Attention mechanism을 적용한 Neural Machine Translation 모델을 사용하여 데이터 클렌징, 사전 크기 변경을 통한 모델 최적화를 진행 하였고, 최적화된 모델에 숫자 기호화 알고리즘을 적용하여 상기 문제점을 해결하는 번역 시스템을 구현하였다. 본논문은 데이터 클렌징 방법과 사전 크기 변경, 그리고 숫자 기호화 알고리즘에 대해 서술하였으며, BLEU score (Bilingual Evaluation Understudy score) 를 이용하여 각 모델의 성능을 비교하였다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. Y. Koren, R. Bell, and C. Volinsky, "Matrix factorization techniques for recommender systems," IEEE Computer Society, Vol. 42, No. 8, pp.30-37, August 2009.
  2. H. Choi, Y. Kang, and M. Kang, "Pet shop recommendation system based on implicit feedback," Journal of Digital Contents Society, Vol. 18, No. 1, pp. 1-4, 2017. https://doi.org/10.9728/dcs.2017.18.1.1
  3. Y. LeCun, Y. Bengio, and G. E. Hinton, "Deep learning," Nature, Vol. 521, No. 7553, pp. 436-444, 2015. https://doi.org/10.1038/nature14539
  4. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Neural Information Processing Systems, pp. 1097-1105, 2012.
  5. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbur, "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal processing magazine, pp.82-97, 2012.
  6. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio, "Show, attend and tell: Neural Image Caption Generation with Visual Attention," in Proceeding of the International Conference on Machine Learning, Lille: France, pp. 2048-2057, July 2015.
  7. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to Sequence Learning with Neural Networks", NIPS 2014.
  8. D. Bahdanau, K. Cho, and Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate", ICLR 2015.
  9. R. Sennrich, B. Haddow, and A. Birch. " Neural machine translation ofrare words with subword units," arXivpreprint arXiv:1508.07909, 2015.
  10. H. Choi, K. Cho, and Y. Bengio, "Context-dependent word representation for neural machine translation", Computer Speech & Language, Vol. 45, p. 149-160, 2017. https://doi.org/10.1016/j.csl.2017.01.007

Cited by

  1. Neural Machine Translation with Word Embedding Transferred from Language Model vol.20, pp.11, 2019, https://doi.org/10.9728/dcs.2019.20.11.2211
  2. Autonomous Parking Simulator for Reinforcement Learning vol.21, pp.2, 2018, https://doi.org/10.9728/dcs.2020.21.2.381
  3. Utilizing Machine Translation Systems to Generate Word Lists for Learning Vocabulary in English vol.22, pp.1, 2021, https://doi.org/10.9728/dcs.2021.22.1.71