Journal of Convergence for Information Technology (융합정보논문지)

Convergence Society for SMB (중소기업융합학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Accuracy and Loss Performance According to Hyperparameter in RNN Model

RNN모델에서 하이퍼파라미터 변화에 따른 정확도와 손실 성능 분석

  • Kim, Joon-Yong (Department of IT Convergence Software, Seoul Theological University) ;
  • Park, Koo-Rack (Division of Computer Science & Engineering, Kongju National University)
  • 김준용 (서울신학대학교 IT융합소프트웨어학과) ;
  • 박구락 (공주대학교 컴퓨터공학부)
  • Received : 2021.06.01
  • Accepted : 2021.07.20
  • Published : 2021.07.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, in order to obtain the optimization of the RNN model used for sentiment analysis, the correlation of each model was studied by observing the trend of loss and accuracy according to hyperparameter tuning. As a research method, after configuring the hidden layer with LSTM and the embedding layer that are most optimized to process sequential data, the loss and accuracy of each model were measured by tuning the unit, batch-size, and embedding size of the LSTM. As a result of the measurement, the loss was 41.9% and the accuracy was 11.4%, and the trend of the optimization model showed a consistently stable graph, confirming that the tuning of the hyperparameter had a profound effect on the model. In addition, it was confirmed that the decision of the embedding size among the three hyperparameters had the greatest influence on the model. In the future, this research will be continued, and research on an algorithm that allows the model to directly find the optimal hyperparameter will continue.

본 논문은 감성 분석에 사용되는 RNN 모델의 최적화를 얻기 위한 성능분석을 위하여 하이퍼파라미터 튜닝에 따른 손실과 정확도의 추이를 관찰하여 모델과의 상관관계를 연구하였다. 연구 방법으로는 시퀀셜데이터를 처리하는데 가장 최적화된 LSTM과 Embedding layer로 히든레이어를 구성한 후, LSTM의 Unit과 Batch Size, Embedding Size를 튜닝하여 각각의 모델에 대한 손실과 정확도를 측정하였다. 측정 결과, 손실은 41.9%, 정확도는 11.4%의 차이를 나타내었고, 최적화 모델의 변화추이는 지속적으로 안정적인 그래프를 보여 하이퍼파라미터의 튜닝이 모델에 지대한 영향을 미침을 확인하였다. 또한 3가지 하이퍼파라미터 중 Embedding Size의 결정이 모델에 가장 큰 영향을 미침을 확인하였다. 향후 이 연구를 지속적으로 이어나가 모델이 최적의 하이퍼파라미터를 직접 찾아낼 수 있는 알고리즘에 대한 연구를 지속적으로 이어나갈 것이다.

Keywords

References

  1. G. Y. Lee & S. B. Lee. (2018). Universal Prediction System Realization Using RNN. The Journal of Korean Institute of Information Technology, 16(10), 11-20. DOI : 10.14801/jkiit.2018.16.10.11
  2. C. Y. Lee & T. G. Lee, Kyungseop Shin. (2019). Performance Comparison of Natural Language Processing Model Based on Deep Neural Networks. The Journalof Korean Institute Comunications and Information Sciences, 44(7), 1344-1350 https://doi.org/10.7840/kics.2019.44.7.1344
  3. C. Y. Lee & J. Kim, (2018). The prediction and analysis of the power energy time series by using the elman recurrent neural network. Journal of the Society of Korea Industrial and Systems Engineering, 41(1), 84-93. DOI : 10.11627/jkise.2018.41.1.084
  4. Y. Kim et al. (2018). Performance evaluation of machine learning and deep learning algorithms in crop classification: Impact of hyperparameters and training sample size. Korean Journal of Remote Sensing, 34(5), 811-827. DOI : 10.7780/kjrs.2018.34.5.9
  5. I. Aliyu, R. M. Mahmood & C. G. Lim. (2019). LSTM Hyperparameter Optimization for an EEG-Based Efficient Emotion Classification in BCI. The Journal of the Korea institute of electronic communication sciences, 14(6), 1171-1180. DOI : 10.37727/jkdas.2019.21.4.1771
  6. S. Karita et al. (2019, December). A comparative study on transformer vs rnn in speech applications. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) (pp. 449-456). IEEE. DOI : 10.1109/ASRU46091.2019.9003750
  7. K. Rao, H. Sak & R. Prabhavalkar. (2017, December). Exploring architectures, data and units for streaming end-to-end speech recognition with rnn-transducer. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) (pp. 193-199). IEEE. DOI : 10.1109/ASRU.2017.8268935
  8. Colah' Blog. (2015). Understanding LSTM Networks. (Online). https://colah.github.io/posts/2015-08-Understanding-LSTMs/
  9. X. Zhang, F. Chen & R. Huang. (2018). A combination of RNN and CNN for attentionbased relation classification. Procedia computer science, 131, 911-917. https://doi.org/10.1016/j.procs.2018.04.221
  10. C. Baziotis, N. Pelekis & C. Doulkeridis. (2017, August). Datastories at semeval-2017 task 6: Siamese LSTM with attention for humorous text comparison. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) (pp. 390-395).
  11. D. A. Reddy, M. A. Kumar & K. P. Soman. (2019). c In Soft Computing and Signal Processing (pp. 385-394). Springer, Singapore.
  12. Z. Li, R. Kulhanek, S. Wang, Y. Zhao & S. Wu. (2018, April). Slim embedding layers for recurrent neural language models. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 32, No. 1). DOI : 10.1007/978-981-13-3393-4_40
  13. T. Mikolov, I. Sutskever, K. Chen, G. Corrado & J. Dean. (2013). Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546.
  14. N. Reimers & I. Gurevych. (2019). Alternative weighting schemes for elmo embeddings. arXiv preprint arXiv:1904.02954.
  15. I. Santos, N. Nedjah & L. de Macedo Mourelle. (2017, November). Sentiment analysis using convolutional neural network with fastText embeddings. In 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI) (pp. 1-5). IEEE. DOI : 10.1109/LA-CCI.2017.8285683
  16. A. Tifrea, G.Becigneul & O. E. Ganea. (2018). Poincar\'e GloVe: Hyperbolic Word Embeddings. arXiv preprint arXiv:1810.06546.
  17. S. Choi, J. Seol & S. G. Lee. (2016), On Word Embedding Models and Parameters Optimized for Korean. Korean Language Information Science Society, 252-256.