Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Impact of Topic Distribution on Review Sentiment: A Comparative Study between South Korea and the U.S.

  • Mina Cho (Graduate School of Business, Seoul National University) ;
  • Dugmee Hwang (Graduate School of Business, Seoul National University) ;
  • SeongMin Jeon (College of Business, Gachon University)
  • Received : 2022.04.21
  • Accepted : 2022.08.16
  • Published : 2022.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Online reviews offer valuable information to businesses by reflecting consumer experiences about their products and services. Two crucial aspects of online reviews are the topics consumers choose to address, and the sentiments expressed in their reviews. Building upon previous literature that shows online reviews are context-dependent, we employ the Expectation-Confirmation Theory (ECT) to examine the impact of topic distribution on review sentiment in South Korea and the U.S. during pre- and post-pandemic periods. After applying a topic modeling to Airbnb app review data, we measure the contribution of each topic on review sentiment using SHAP values. Our results indicate variations in topic distribution trends between 2018 and 2021. In addition, the order and magnitude of topics' impact on review sentiment change between pre- and post-pandemic periods for both countries. This study can help businesses understand how topics and sentiments associated with their products and services changed after the pandemic and thus identify areas of improvement.

Keywords

References

  1. Abril, D. (2020). Airbnb's IPO filing reveals huge COVID impact. Fortune. Retrieved from https://fortune.com/2020/11/16/airbnb-ipo-initial-public-offering-coronavirus-impact/
  2. Alhojely, S. (2016). Sentiment analysis and opinion mining: A survey. International Journal of Computer Applications, 150(6), 22-25. https://doi.org/10.5120/ijca2016911545 
  3. Ayanso, A., Herath, T. C., and O'Brien, N. (2015). Understanding continuance intentions of physicians with electronic medical records (EMR): An expectancy-confirmation perspective. Decision Support Systems, 77, 112-122. https://doi.org/10.1016/j.dss.2015.06.003 
  4. Baptista, M. L., Goebel, K., and Henriques, E. M. P. (2022). Relation between prognostics predictor evaluation metrics and local interpretability SHAP values. Artificial Intelligence, 306, 103667. https://doi.org/10.1016/j.artint.2022.103667 
  5. Bastani, K., Namavari, H., and Shaffer, J. (2019). Latent Dirichlet allocation (LDA) for topic modeling of the CFPB consumer complaints. Expert Systems with Applications, 127, 256-271. https://doi.org/10.48550/arXiv.1807.07468 
  6. Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25(3), 351-370. https://doi.org/10.2307/3250921 
  7. Bickart, B., and Schindler, R. M. (2001). Internet forums as influential sources of consumer information. Journal of Interactive Marketing, 15(3), 31-40. https://doi.org/10.1002/dir.1014 
  8. Blei, D. M., Ng, A. Y., and Edu, J. B. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993-1022. 
  9. de Oliveira Capela, F., and Ramirez-Marquez, J. E. (2019). Detecting urban identity perception via newspaper topic modeling. Cities, 93, 72-83. https://doi.org/10.1016/j.cities.2019.04.009 
  10. Esuli, A., and Sebastiani, F. (2006). SENTIWORDNET: A publicly available lexical resource for opinion mining. Proceedings of the 5th International Conference on Language Resources and Evaluation, LREC 2006, 417-422. Retrieved from http://www.lrec-conf.org/proceedings/lrec2006/pdf/384_pdf.pdf 
  11. Fullerton, L. (2017). Online reviews impact purchasing decisions for over 93% of consumers, report suggests, Retrieved from https://www.thedrum.com/news/2017/03/27/online-reviews-impact-purchasing-decisions-over-93-consumers-report-suggests 
  12. Futagami, K., Fukazawa, Y., Kapoor, N., and Kito, T. (2021). Pairwise acquisition prediction with SHAP value interpretation. Journal of Finance and Data Science, 7, 22-44. https://doi.org/10.1016/j.jfds.2021.02.001 
  13. Geetha, M., Singha, P., and Sinha, S. (2017). Relationship between customer sentiment and online customer ratings for hotels: An empirical analysis. Tourism Management, 61, 43-54. https://doi.org/10.1016/j.tourman.2016.12.022 
  14. Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., and Yang, G. Z. (2019). XAI-Explainable artificial intelligence. Science Robotics, 4(37). https://doi.org/10.1126/scirobotics.aay7120 
  15. Guzman, E., Azocar, D., and Li, Y. (2014). Sentiment analysis of commit comments in GitHub: An empirical study. 11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings, 352-355. https://doi.org/10.1145/2597073.2597118 
  16. Hasan, M., Rahman, A., Karim, M. R., Khan, M. S. I., and Islam, M. J. (2021). Normalized approach to find optimal number of topics in Latent Dirichlet Allocation (LDA). Advances in Intelligent Systems and Computing, 1309, 341-354. https://doi.org/10.1007/978-981-33-4673-4_27 
  17. Hong, L., and Davison, B. D. (2010). Empirical study of topic modeling in Twitter. SOMA 2010 - Proceedings of the 1st Workshop on Social Media Analytics, 80-88. https://doi.org/10.1145/1964858.1964870 
  18. Hsupeng, B., Lee, K. W., Wei, T. E., and Wang, S. H. (2022). Explainable malware detection using predefined network flow. 2022 24th International Conference on Advanced Communication Technology (ICACT), 27-33. https://doi.org/10.23919/ICACT53585.2022.9728897 
  19. Hu, F., Teichert, T., Deng, S., Liu, Y., and Zhou, G. (2021). Dealing with pandemics: An investigation of the effects of COVID-19 on customers' evaluations of hospitality services. Tourism Management, 85, 104320. https://doi.org/10.1016/j.tourman.2021.104320 
  20. Hu, N., Bose, I., Koh, N. S., and Liu, L. (2012). Manipulation of online reviews: An analysis of ratings, readability, and sentiments. Decision Support Systems, 52(3), 674-684. https://doi.org/10.1016/j.dss.2011.11.002 
  21. Hu, Y., Boyd-Graber, J., Satinoff, B., and Smith, A. (2014). Interactive topic modeling. Machine Learning, 95(3), 423-469. https://doi.org/10.1007/s10994-013-5413-0 
  22. Huang, A. H., Lehavy, R., Zang, A. Y., and Zheng, R. (2018). Analyst information discovery and interpretation roles: A topic modeling approach. Management Science, 64(6), 2833-2855. https://doi.org/10.1287/mnsc.2017.2751 
  23. Hutto, C. J., and Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the 8th International Conference on Weblogs and Social Media, ICWSM 2014, 216-225. Retrieved from https://ojs.aaai.org/index.php/ICWSM/article/view/14550 
  24. Ilyas, S. H. W., Soomro, Z. T., Anwar, A., Shahzad, H., and Yaqub, U. (2020). Analyzing brexit's impact using sentiment analysis and topic modeling on twitter discussion. ACM International Conference Proceeding Series, 1-6. https://doi.org/10.1145/3396956.3396973 
  25. Jeong, S. Y., Kim, J. W., Kim, Y. S., Joo, H. Y., and Moon, J. H. (2021). Sentiment analysis of nuclear energy-related articles and their comments on a portal site in Rep. of Korea in 2010-2019. Nuclear Engineering and Technology, 53(3), 1013-1019. https://doi.org/10.1016/j.net.2020.07.031 
  26. Khanthaapha, P., Pipanmaekaporn, L., and Kamonsantiroj, S. (2018). Topic-based user profile model for POI recommendations. ACM International Conference Proceeding Series, 143-147. https://doi.org/10.1145/3206185.3206203 
  27. Kim, E. H. J., Jeong, Y. K., Kim, Y., Kang, K. Y., and Song, M. (2016). Topic-based content and sentiment analysis of Ebola virus on Twitter and in the news. Journal of Information Science, 42(6), 763-781. https://doi.org/10.1177/0165551515608733 
  28. Kwon, H. J., Ban, H. J., Jun, J. K., and Kim, H. S. (2021). Topic modeling and sentiment analysis of online review for airlines. Information 2021, 12(2), 78. https://doi.org/10.3390/info12020078 
  29. Lee, C. K. H. (2022). How guest-host interactions affect consumer experiences in the sharing economy: New evidence from a configurational analysis based on consumer reviews. Decision Support Systems, 152, 113634. https://doi.org/10.1016/j.dss.2021.113634 
  30. Lee, H., Noh, E. B., Park, S. J., Nam, H. K., Lee, T. H., Lee, G. R., and Nam, E. W. (2021). COVID-19 vaccine perception in South Korea: Web crawling approach. JMIR Public Health and Surveillance, 7(9), e31409. https://doi.org/10.2196/31409 
  31. Lee, M., Jeong, M., and Lee, J. (2017). Roles of negative emotions in customers' perceived helpfulness of hotel reviews on a user-generated review website: A text mining approach. International Journal of Contemporary Hospitality Management, 29(2), 762-783. https://doi.org/10.1108/IJCHM-10-2015-0626 
  32. Liao, C., Lin, H. N., Luo, M. M., and Chea, S. (2017). Factors influencing online shoppers' repurchase intentions: The roles of satisfaction and regret. Information & Management, 54(5), 651-668. https://doi.org/10.1016/j.im.2016.12.005 
  33. Liu, A. X., Li, Y., and Xu, S. X. (2021). Assessing the unacquainted: Inferred reviewer personality and review helpfulness. MIS Quarterly: Management Information Systems, 45(3), 1113-1148. https://doi.org/10.25300/MISQ/2021/14375 
  34. Lu, Y., Hu, X., Wang, F., Kumar, S., Liu, H., and Maciejewski, R. (2015). Visualizing social media sentiment in disaster scenarios. WWW 2015 Companion - Proceedings of the 24th International Conference on World Wide Web, 1211-1215. https://doi.org/10.1145/2740908.2741720 
  35. Lu, Y., and Zheng, Q. (2021). Twitter public sentiment dynamics on cruise tourism during the COVID-19 pandemic. Current Issues in Tourism, 24(7), 892-898. https://doi.org/10.1080/13683500.2020.1843607 
  36. Lundberg, S. M., and Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 2017-Dec., 4766-4775. https://doi.org/10.48550/arXiv.1705.07874 
  37. Maheswaran, D., Mackie, D. M., and Chaiken, S. (1992). Brand name as a heuristic cue: The effects of task importance and expectancy confirmation on consumer judgments. Journal of Consumer Psychology, 1(4), 317-336. https://doi.org/10.1016/S1057-7408(08)80058-7 
  38. McIlroy, S., Ali, N., Khalid, H., and E. Hassan, A. (2016). Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empirical Software Engineering, 21(3), 1067-1106. https://doi.org/10.1007/s10664-015-9375-7 
  39. Mehta, M. P., Kumar, G., and Ramkumar, M. (2021). Customer expectations in the hotel industry during the COVID-19 pandemic: A global perspective using sentiment analysis. Tourism Recreation Research. https://doi.org/10.1080/02508281.2021.1894692 
  40. Mohammed, S. H., and Al-Augby, S. (2020). LSA & LDA topic modeling classification: Comparison study on E-books. Indonesian Journal of Electrical Engineering and Computer Science, 19(1), 353-362. http://doi.org/10.11591/ijeecs.v19.i1.pp353-362 
  41. Mostafa, M. M. (2013). More than words: Social networks' text mining for consumer brand sentiments. Expert Systems with Applications, 40(10), 4241-4251. https://doi.org/10.1016/j.eswa.2013.01.019 
  42. Nikolenko, S. I., Koltcov, S., and Koltsova, O. (2017). Topic modelling for qualitative studies. Article Journal of Information Science, 43(1), 88-102. https://doi.org/10.1177/016555151561739 
  43. Ohana, B., and Tierney, B. (2009). Sentiment classification of reviews using SentiWordNet. 9th. IT & T Conference. https://doi.org/10.21427/D77S56 
  44. Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460. https://doi.org/10.1177/002224378001700405 
  45. Ramteke, J., Shah, S., Godhia, D., and Shaikh, A. (2016). Election result prediction using Twitter sentiment analysis. Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016, 1. https://doi.org/10.1109/INVENTIVE.2016.7823280 
  46. Saifee, D. H., Zheng, Z., Bardhan, I. R., and Lahiri, A. (2020). Are online reviews of physicians reliable indicators of clinical outcomes? A focus on chronic disease management. Information Systems Research, 31(4), 1282-1300. https://doi.org/10.1287/isre.2020.0945 
  47. Shim, J. G., Ryu, K. H., Lee, S. H., Cho, E. A., Lee, Y. J., and Ahn, J. H. (2021). Text mining approaches to analyze public sentiment changes regarding covid-19 vaccines on social media in korea. International Journal of Environmental Research and Public Health, 18(12), 6549. https://doi.org/10.3390/ijerph18126549 
  48. Shin, D. H. (2011). Understanding e-book users: Uses and gratification expectancy model. New Media and Society, 13(2), 260-278. https://doi.org/10.1177/1461444810372163 
  49. Sirbu, D., Secui, A., Dascalu, M., Crossley, S. A., Ruseti, S., and Trausan-Matu, S. (2017). Extracting gamers' opinions from reviews. Proceedings - 18th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2016, 227-232. http://doi.org/10.1109/SYNASC.2016.044 
  50. Stieglitz, S., and Dang-Xuan, L. (2013). Emotions and information diffusion in social media - Sentiment of microblogs and sharing behavior. Journal of Management Information Systems, 29(4), 217-248. https://doi.org/10.2753/MIS0742-1222290408 
  51. Sutherland, I., and Kiatkawsin, K. (2020). Determinants of guest experience in Airbnb: A topic modeling approach using LDA. Sustainability (Switzerland), 12(8). https://doi.org/10.3390/su12083402 
  52. Sutherland, I., Sim, Y., Lee, S. K., Byun, J., and Kiatkawsin, K. (2020). Topic modeling of online accommodation reviews via latent dirichlet allocation. Sustainability (Switzerland), 12(5), 1-15. https://doi.org/10.3390/su12051821 
  53. Syed, S., and Spruit, M. (2017). Full-Text or abstract? Examining topic coherence scores using latent dirichlet allocation. 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017, 2018-Jan., 165-174. http://doi.org/10.1109/DSAA.2017.61 
  54. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., and Stede, M. (2011). Lexicon-based methods for sentiment analysis. Computational Linguistics, 37(2), 267-307. https://doi.org/10.1162/COLI_a_00049 
  55. Thelwall, M. (2017). The Heart and Soul of the Web? Sentiment Strength Detection in the Social Web with SentiStrength (pp. 119-134). Springer, Cham. 
  56. Thomsen, C. (R.), and Jeong, M. (2021). An analysis of Airbnb online reviews: user experience in 16 U.S. cities. Journal of Hospitality and Tourism Technology, 12(1), 97-111. https://doi.org/10.1108/JHTT-02-2019-0023 
  57. Wang, L., and Kirilenko, A. P. (2021). Do tourists from different countries interpret travel experience with the same feeling? sentiment analysis of tripadvisor reviews. In Information and Communication Technologies in Tourism 2021 (pp. 294-301). Springer, Cham. https://doi.org/10.1007/978-3-030-65785-7_27 
  58. Xiang, Z., Du, Q., Ma, Y., and Fan, W. (2017). A comparative analysis of major online review platforms: Implications for social media analytics in hospitality and tourism. Tourism Management, 58, 51-65. https://doi.org/10.1016/j.tourman.2016.10.001 
  59. Zhang, J. (2019). Listening to the consumer: exploring review topics on airbnb and their impact on listing performance. Journal of Marketing Theory and Practice, 27(4), 371-389. https://doi.org/10.1080/10696679.2019.1644953 
  60. Zhang, J., and Piramuthu, S. (2018). Product recommendation with latent review topics. Information Systems Frontiers, 20(3), 617-625. https://doi.org/10.1007/s10796-016-9697-z 
  61. Zhang, K., Xu, P., and Zhang, J. (2020). Explainable AI in deep reinforcement learning models: A SHAP method applied in power system emergency control. 2020 IEEE 4th Conference on Energy Internet and Energy System Integration: Connecting the Grids Towards a Low-Carbon High-Efficiency Energy System, EI2 2020, 711-716. http://doi.org/10.1109/TCSS.2021.3096824 
  62. Zhang, L., Yan, Q., and Zhang, L. (2020). A text analytics framework for understanding the relationships among host self-description, trust perception and purchase behavior on Airbnb. Decision Support Systems, 133, 113288. https://doi.org/10.1016/j.dss.2020.113288 
  63. Zhu, B., Zheng, X., Liu, H., Li, J., and Wang, P. (2020). Analysis of spatiotemporal characteristics of big data on social media sentiment with COVID-19 epidemic topics. Chaos, Solitons and Fractals, 140, 110123. https://doi.org/10.1016/j.chaos.2020.110123