Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Cross-Domain Text Sentiment Classification Method Based on the CNN-BiLSTM-TE Model

  • Zeng, Yuyang (School of Business, Sichuan Agricultural University) ;
  • Zhang, Ruirui (School of Business, Sichuan Agricultural University) ;
  • Yang, Liang (School of Business, Sichuan Agricultural University) ;
  • Song, Sujuan (School of Business, Sichuan Agricultural University)
  • Received : 2020.07.27
  • Accepted : 2020.10.19
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

To address the problems of low precision rate, insufficient feature extraction, and poor contextual ability in existing text sentiment analysis methods, a mixed model account of a CNN-BiLSTM-TE (convolutional neural network, bidirectional long short-term memory, and topic extraction) model was proposed. First, Chinese text data was converted into vectors through the method of transfer learning by Word2Vec. Second, local features were extracted by the CNN model. Then, contextual information was extracted by the BiLSTM neural network and the emotional tendency was obtained using softmax. Finally, topics were extracted by the term frequency-inverse document frequency and K-means. Compared with the CNN, BiLSTM, and gate recurrent unit (GRU) models, the CNN-BiLSTM-TE model's F1-score was higher than other models by 0.0147, 0.006, and 0.0052, respectively. Then compared with CNN-LSTM, LSTM-CNN, and BiLSTM-CNN models, the F1-score was higher by 0.0071, 0.0038, and 0.0049, respectively. Experimental results showed that the CNN-BiLSTM-TE model can effectively improve various indicators in application. Lastly, performed scalability verification through a takeaway dataset, which has great value in practical applications.

Keywords

References

  1. J. Li and H. Li, "Research on product feature extraction and sentiment classification of short online review based on deep learning," Chinese Journal of Information Studies: Theory & Application, vol. 41, no. 2, pp. 143-148, 2018.
  2. J. Qiu, C. Liu, Y. Li, and Z. Lin, "Leveraging sentiment analysis at the aspects level to predict ratings of reviews," Information Sciences, vol. 451, pp. 295-309, 2018. https://doi.org/10.1016/j.ins.2018.04.009
  3. H. Tang, S. Tan, and X. Cheng, "A survey on sentiment detection of reviews," Expert Systems with Applications, vol. 36, no. 7, pp. 10760-10773, 2009. https://doi.org/10.1016/j.eswa.2009.02.063
  4. J. F. Xu, Y. Xu, Y. C. Xu, Y. Zhang, and Q. Liu, "Hybrid algorithm framework for sentiment classification of chinese based on semantic comprehension and machine learning," Computer Science, vol. 42, no. 6, pp. 61-66, 2015. https://doi.org/10.11896/j.issn.1002-137X.2015.6.014
  5. A. Joshi, A. R. Balamurali, P. Bhattacharyya, and R. Mohanty, "C-Feel-It: a sentiment analyzer for microblogs," in Proceedings of the ACL-HLT 2011 System Demonstrations, Portland, OR, 2011, pp. 127-132.
  6. E. Boiy and M. F. Moens, "A machine learning approach to sentiment analysis in multilingual web texts," Information Retrieval, vol. 12, no. 5, pp. 526-558, 2009. https://doi.org/10.1007/s10791-008-9070-z
  7. Y. X. He, S. T. Sun, F. F. Niu, and F. Li, "A deep learning model enhanced with emotion semantics for microblog sentiment analysis," Chinese Journal of Computers, vol. 40, no. 4, pp. 773-790, 2017.
  8. G. A. Miller, "WordNet: a lexical database for English," Communications of the ACM, vol. 38, no. 11, pp. 39-41, 1995. https://doi.org/10.1145/219717.219748
  9. L. W. Ku and H. H. Chen, "Mining opinions from the web: beyond relevance retrieval," Journal of the American Society for Information Science and Technology, vol. 58, no. 12, pp. 1838-1850, 2007. https://doi.org/10.1002/asi.20630
  10. P. D. Turney and P. Pantel, "From frequency to meaning: vector space models of semantics," Journal of Artificial Intelligence Research, vol. 37, pp. 141-188, 2010. https://doi.org/10.1613/jair.2934
  11. B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up? Sentiment classification using machine learning techniques," in Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), Philadelphia, PA, 2002, pp. 79-86.
  12. S. Tan and J. Zhang, "An empirical study of sentiment analysis for Chinese documents," Expert Systems with Applications, vol. 34, no. 4, pp. 2622-2629, 2008. https://doi.org/10.1016/j.eswa.2007.05.028
  13. N. Kalchbrenner, E. Grefenstette, and P. Blunsom, "A convolutional neural network for modelling sentences," in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, MD, 2014, pp. 655-665.
  14. Y. Kim, "Convolutional neural networks for sentence classification," in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014, pp. 1746-1751.
  15. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  16. J. Liang, Y. Chai, H. Yuan, M. L. Gao, and H. Zan, "Polarity shifting and LSTM based recursive networks for sentiment analysis," Journal of Chinese Information Processing, vol. 29, no. 5, pp. 152-159, 2015.
  17. A. Graves, N. Jaitly, and A. R. Mohamed, "Hybrid speech recognition with deep bidirectional LSTM," in Proceedings of 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic, 2013, pp. 273-278.
  18. S. F. Zeng, X. Y. Zhang, X. F. Du, and T. B. Lu, "New method of text representation model based on neural network," Journal on Communications, vol. 38, no. 4, pp. 86-98, 2017.
  19. G. E. Hinton, "Learning distributed representations of concepts," in Proceedings of the 8th Annual Conference of the Cognitive Science Society, Amherst, MA, 1986.
  20. Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin, "A neural probabilistic language model," The Journal of Machine Learning Research, vol. 3, pp. 1137-1155, 2003.
  21. T. Mikolov, K Chen, G Corrado, and J. Dean. "Efficient estimation of word representations in vector space," 2012 [Online]. Available: https://arxiv.org/abs/1301.3781.
  22. X. D. Guo, N. Zhao, and S. Z. Cui, "Consumer reviews sentiment analysis based on CNN-BiLSTM," Systems Engineering-Theory & Practice, vol. 40, pp. 653-663, 2020.
  23. S. P. Zhai and Y. Y. Yang, "Bilingual text sentiment analysis based on attention mechanism Bi-LSTM," Computer Applications and Software, vol. 36, no. 12, pp. 251-255, 2019.
  24. J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, and G. Zhang, "Transfer learning using computational intelligence: a survey," Knowledge-Based Systems, vol. 80, pp. 14-23, 2015. https://doi.org/10.1016/j.knosys.2015.01.010
  25. G. E. Hinton, N. Srivastava, and A. Krizhevsky, "Improving neural networks by preventing co-adaption of feature detectors," 2012 [Online]. Available: https://arxiv.org/abs/1207.0580.
  26. Y. Zhang, Y. Jiang, and Y. Tong, "Study of sentiment classification for Chinese microblog based on recurrent neural network," Chinese Journal of Electronics, vol. 25, no. 4, pp. 601-607, 2016. https://doi.org/10.1049/cje.2016.07.002
  27. H. Zhao, L. Wang, and W. Wang, "Text sentiment analysis based on serial hybrid model of bi-directional long short-term memory and convolutional neural network," Journal of Computer Applications, vol. 40, no. 1, pp. 16-20, 2020.