The Journal of Information Systems (한국정보시스템학회지:정보시스템연구)

Korea Association of Information Systems (한국정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Customer Service Evaluation based on Online Text Analytics: Sentiment Analysis and Structural Topic Modeling

  • Park, KyungBae (School of Business Administration, Kyungpook National University) ;
  • Ha, Sung Ho (School of Business Administration, Kyungpook National University)
  • Received : 2017.11.28
  • Accepted : 2017.12.11
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Purpose Social media such as social network services, online forums, and customer reviews have produced a plethora amount of information online. Yet, the information deluge has created both opportunities and challenges at the same time. This research particularly focuses on the challenges in order to discover and track the service defects over time derived by mining publicly available online customer reviews. Design/methodology/approach Synthesizing the streams of research from text analytics, we apply two stages of methods of sentiment analysis and structural topic model incorporating meta-information buried in review texts into the topics. Findings As a result, our study reveals that the research framework effectively leverages textual information to detect, prioritize, and categorize service defects by considering the moving trend over time. Our approach also highlights several implications theoretically and practically of how methods in computational linguistics can offer enriched insights by leveraging the online medium.

Keywords

References

  1. Archak, N., Ghose, A., and Ipeirotis, P.G., "Deriving the pricing power of product features by mining consumer reviews," Management Science, Vol. 57, 2011, pp. 1485-1509. https://doi.org/10.1287/mnsc.1110.1370
  2. Bickart, B., and Schindler, R.M., "Internet forums as influential sources of consumer information," Journal of Interactive Marketing, Vol. 15, 2001, pp. 31-40. https://doi.org/10.1002/dir.1014
  3. Blei, D.M., "Probabilistic topic models," Communications of the ACM, Vol. 55, 2012, pp. 77-84. https://doi.org/10.1145/2133806.2133826
  4. Blei, D.M., and Lafferty, J.D., "A correlated topic model of science," The Annals of Applied Statistics, Vol. 1, No. 1, 2007, pp. 17-35. https://doi.org/10.1214/07-AOAS114
  5. Blei, D.M., and Lafferty, J.D., "Dynamic topic models," in: Proceedings of the 23rd ACM International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA, 2006, pp. 113-120.
  6. Blei, D.M., Ng, A.Y., and Jordan, M.I., "Latent dirichlet allocation," Journal of Machine Learning Research, Vol. 3, 2003, pp. 993-1022.
  7. Bollen, J., Mao, H., and Pepe, A., "Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena," in: Proceedings of Fifth International AAAI Conference on Weblogs and Social Media, 2009.
  8. Bollen, J., Mao, H., and Zeng, X., "Twitter mood predicts the stock market," Journal of Computational Science, Vol. 2, 2011, pp. 1-8. https://doi.org/10.1016/j.jocs.2010.12.007
  9. Browning, V., So, K.K.F., and Sparks, B., "The influence of online reviews on consumers' attributions of service quality and control for service standards in hotels," Journal of Travel & Tourism Marketing, Vol. 30, 2013, pp. 23-40. https://doi.org/10.1080/10548408.2013.750971
  10. Cantallops, A.S., and Salvi, F., "New consumer behavior: A review of research on eWOM and hotels," International Journal of Hospitality Management, Vol. 36, 2014, pp. 41-51. https://doi.org/10.1016/j.ijhm.2013.08.007
  11. Chevalier, J.A., and Mayzlin, D., "The effect of word of mouth on sales: Online book reviews," Journal of Marketing Research, Vol. 43, 2006, pp. 345-354. https://doi.org/10.1509/jmkr.43.3.345
  12. Dellarocas, C., Zhang, X.M., and Awad, N.F., "Exploring the value of online product reviews in forecasting sales: The case of motion pictures," Journal of Interactive Marketing, Vol. 21, 2007, pp. 23-45. https://doi.org/10.1002/dir.20087
  13. Edmunds, A., and Morris, A., "The problem of information overload in business organisations: a review of the literature," International Journal of Information Management, Vol. 20, 2000, pp. 17-28. https://doi.org/10.1016/S0268-4012(99)00051-1
  14. Finch, B.J., "Internet discussions as a source for consumer product customer involvement and quality information: an exploratory study," Journal of Operations Management, Vol. 17, 1999, pp. 535-556. https://doi.org/10.1016/S0272-6963(99)00005-4
  15. Ghose, A., Ipeirotis, P.G., and Li, B., "Designing ranking systems for hotels on travel search engines by mining user-generated and crowdsourced content," Marketing Science, Vol. 31, 2012, pp. 493-520. https://doi.org/10.1287/mksc.1110.0700
  16. Gopal, R., Marsden, J.R., and Vanthienen, J., "Information mining-Reflections on recent advancements and the road ahead in data, text, and media mining," Decision Support Systems, Vol. 51, 2011, pp. 727-731. https://doi.org/10.1016/j.dss.2011.01.008
  17. Griffiths, T.L., Jordan, M.I., Tenenbaum, J.B., and Blei, D.M., "Hierarchical topic models and the nested Chinese restaurant process," in: Advances in Neural Information Processing Systems, 2004, pp. 17-24.
  18. Griffiths, T.L., and Steyvers, M., "Finding scientific topics," in: Proceedings of the National Academy of Sciences of the United States of America, 2004, pp. 5228-5235.
  19. Grimmer, J., and Stewart, B.M., "Text as data: The promise and pitfalls of automatic content analysis methods for political texts," Political Analysis, Vol. 21, 2013, pp. 267-297. https://doi.org/10.1093/pan/mps028
  20. Guo, Y., Barnes, S.J., and Jia, Q., "Mining meaning from online ratings and reviews: Tourist satisfaction analysis using latent dirichlet allocation," Tourism Management, Vol. 59, 2017, pp. 467-483. https://doi.org/10.1016/j.tourman.2016.09.009
  21. Kim, I.S., "Political cleavages within industry: firm-level lobbying for trade liberalization," American Political Science Review, Vol. 111, 2017, pp. 1-20. https://doi.org/10.1017/S0003055416000654
  22. Li, X., and Hitt, L.M., "Self-selection and information role of online product reviews," Information Systems Research, Vol. 19, 2008, pp. 456-474. https://doi.org/10.1287/isre.1070.0154
  23. Litvin, S.W., Goldsmith, R.E., and Pan, B., "Electronic word-of-mouth in hospitality and tourism management," Tourism Management, Vol. 29, 2008, pp. 458-468. https://doi.org/10.1016/j.tourman.2007.05.011
  24. Liu, B., "Sentiment analysis and opinion mining," Synthesis Lectures on Human Language Technologies, Vol. 5, 2012, pp. 1-167.
  25. Liu, B., Sentiment Analysis: Mining Opinions, Sentiments, and Emotions, Cambridge University Press, 2015.
  26. Lucas, C., Nielsen, R.A., Roberts, M.E., Stewart, B.M., Storer, A., and Tingley, D., "Computer-assisted text analysis for comparative politics," Political Analysis, Vol. 23, 2015, pp. 254-277. https://doi.org/10.1093/pan/mpu019
  27. Martin, F., and Johnson, M., "More efficient topic modelling through a noun only approach," in: Australasian Language Technology Association Workshop, 2015, pp. 111.
  28. McAuley, J., and Leskovec, J., "Hidden factors and hidden topics: understanding rating dimensions with review text," in: Proceedings of the 7th ACM conference on Recommender systems, 2013, pp. 165-172.
  29. Mimno, D., Wallach, H.M., Talley, E., Leenders, M., and McCallum, A., "Optimizing semantic coherence in topic models," in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 2011, pp. 262-272.
  30. Munksgaard, R., and Demant, J., "Mixing politics and crime-The prevalence and decline of political discourse on the cryptomarket," International Journal of Drug Policy, Vol. 35, 2016, pp. 77-83. https://doi.org/10.1016/j.drugpo.2016.04.021
  31. Naveed, N., Gottron, T., Kunegis, J., Alhadi, A.C., "Bad news travel fast: A content-based analysis of interestingness on twitter," in: Proceedings of the 3rd ACM International Web Science Conference, 2011, pp. 8.
  32. Pang, B., and Lee, L., "Opinion mining and sentiment analysis," Foundations and trends in information retrieval, Vol. 2, 2008, pp. 1-135. https://doi.org/10.1561/1500000011
  33. Reich, J., Tingley, D., Leder-Luis, J., Roberts, M.E., and Stewart, B., "Computer-assisted reading and discovery for student generated text in massive open online courses," Journal of Learning Analytics, Vol. 2, 2014, pp. 156-184.
  34. Roberts, M.E., Stewart, B.M., and Airoldi, E.M., "A model of text for experimentation in the social sciences," Journal of the American Statistical Association, Vol. 111, 2016, pp. 988-1003. https://doi.org/10.1080/01621459.2016.1141684
  35. Roberts, M.E., Stewart, B.M., Tingley, D., Airoldi, E.M., "The structural topic model and applied social science," in: Advances in Neural Information Processing Systems Workshop on Topic Models: Computation, Application, and Evaluation, 2013.
  36. Roberts, M.E., Stewart, B.M., Tingley, D., Lucas, C., Leder-Luis, J., Gadarian, S.K., Albertson, B., and Rand, D.G., "Structural topic models for openended survey responses," American Journal of Political Science, Vol. 58, 2014, pp. 1064-1082. https://doi.org/10.1111/ajps.12103
  37. Schofield, A., Magnusson, M., and Mimno, D., "Pulling out the stops: Rethinking stopword removal for topic models," in: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017, pp. 432.
  38. Singh, V., Piryani, R., Uddin, A., and Waila, P., "Sentiment analysis of movie reviews: A new feature-based heuristic for aspect-level sentiment classification," in: IEEE International Multi-Conference on Automation, Computing, Communication, Control and Compressed Sensing(iMac4s), 2013, pp. 712-717.
  39. Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., and Potts, C., "Recursive deep models for semantic compositionality over a sentiment treebank," in: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2013, pp. 1642.
  40. Tirunillai, S., and Tellis, G.J., "Mining marketing meaning from online chatter: Strategic brand analysis of big data using latent dirichlet allocation," Journal of Marketing Research, Vol. 51, 2014, pp. 463-479. https://doi.org/10.1509/jmr.12.0106
  41. Wang, H., Lu, Y., and Zhai, C., "Latent aspect rating analysis on review text data: a rating regression approach," in: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2010, pp. 783-792.
  42. Wu, H.C., and Ko, Y.J., "Assessment of service quality in the hotel industry," Journal of Quality Assurance in Hospitality & Tourism, Vol. 14, 2013, pp. 218-244. https://doi.org/10.1080/1528008X.2013.802557
  43. Xiang, Z., Schwartz, Z., Gerdes, J.H., and Uysal, M., "What can big data and text analytics tell us about hotel guest experience and satisfaction?," International Journal of Hospitality Management, Vol. 44, 2015, pp. 120-130. https://doi.org/10.1016/j.ijhm.2014.10.013
  44. Ye, Q., Law, R., and Gu, B., "The impact of online user reviews on hotel room sales," International Journal of Hospitality Management, Vol. 28, 2009, pp. 180-182. https://doi.org/10.1016/j.ijhm.2008.06.011
  45. Ye, Q., Law, R., Gu, B., and Chen, W., "The influence of user-generated content on traveler behavior: An empirical investigation on the effects of e-word-of-mouth to hotel online bookings," Computers in Human Behavior, Vol. 27, 2011, pp. 634-639. https://doi.org/10.1016/j.chb.2010.04.014
  46. Yu, X., Liu, Y., Huang, X., and An, A., "Mining online reviews for predicting sales performance: A case study in the movie domain," IEEE Transactions on Knowledge and Data Engineering, Vol. 24, 2012, pp. 720-734. https://doi.org/10.1109/TKDE.2010.269
  47. Zhang, W., Xu, H., and Wan, W., "Weakness finder: Find product weakness from Chinese reviews by using aspects based sentiment analysis," Expert Systems with Applications, Vol. 39, 2012, pp. 10283-10291. https://doi.org/10.1016/j.eswa.2012.02.166