Journal of the Korea Convergence Society (한국융합학회논문지)

Korea Convergence Society (한국융합학회)
권호 표지
DOI QR코드

DOI QR Code

Exploring user experience factors through generational online review analysis of AI speakers

인공지능 스피커의 세대별 온라인 리뷰 분석을 통한 사용자 경험 요인 탐색

  • Park, Jeongeun (Graduate School of Information, Yonsei University) ;
  • Yang, Dong-Uk (Graduate School of Information, Yonsei University) ;
  • Kim, Ha-Young (Graduate School of Information, Yonsei University)
  • 박정은 (연세대학교 비즈니스 빅데이터 분석 트랙) ;
  • 양동욱 (연세대학교 비즈니스 빅데이터 분석 트랙) ;
  • 김하영 (연세대학교 비즈니스 빅데이터 분석 트랙)
  • Received : 2021.05.17
  • Accepted : 2021.07.20
  • Published : 2021.07.28
Download PDF
⟨ Previous Next ⟩

Abstract

The AI speaker market is growing steadily. However, the satisfaction of actual users is only 42%. Therefore, in this paper, we collected reviews on Amazon Echo Dot 3rd and 4th generation models to analyze what hinders the user experience through the topic changes and emotional changes of each generation of AI speakers. By using topic modeling analysis techniques, we found changes in topics and topics that make up reviews for each generation, and examined how user sentiment on topics changed according to generation through deep learning-based sentiment analysis. As a result of topic modeling, five topics were derived for each generation. In the case of the 3rd generation, the topic representing general features of the speaker acted as a positive factor for the product, while user convenience features acted as negative factor. Conversely, in the 4th generation, general features were negatively, and convenience features were positively derived. This analysis is significant in that it can present analysis results that take into account not only lexical features but also contextual features of the entire sentence in terms of methodology.

인공지능 스피커 시장은 꾸준히 성장하고 있지만, 실제 스피커 사용자들의 만족도는 42%에 그치고 있다. 따라서, 본 연구에서는 인공지능 스피커의 세대별 토픽 변화와 감성 변화를 통해 사용자 경험을 저해하는 요소는 무엇인지 분석해 보고자 한다. 이를 위해 아마존 에코 닷 3세대와 4세대 모델에 대한 리뷰를 수집하였다. 토픽모델링 분석 기법을 사용하여 세대별로 리뷰를 이루는 주제 및 주제의 변화를 찾아내고, 딥러닝 기반 감성 분석을 통해 토픽에 대한 사용자 감성이 세대에 따라 어떻게 변화되었는지 살펴보았다. 토픽모델링 결과, 세대별로 5개의 토픽이 도출되었다. 3세대의 경우 스피커의 일반적 속성을 나타내는 토픽은 제품에 긍정적 반응 요인으로 작용했고, 사용자 편의 기능은 부정적 반응 요인으로 작용했다. 반대로 4세대에서는 일반적 속성은 부정적으로, 사용자 편의 기능은 긍정적으로 도출되었다. 이와 같은 분석은 방법론 측면에서 어휘적 특징뿐 아니라 문장 전체의 문맥적 특징이 고려된 분석결과를 제시할 수 있다는 것에 그 의의가 있다.

Keywords

Acknowledgement

This research was supported by a grant (NRF-2020R1F1A1071527) from the National Research Foundation of Korea (NRF), funded by the Korea government (MSIT; Ministry of Science and ICT).

References

  1. Market Study Report LLC. (2021). Global Smart Speaker Market Size study, by Intelligent Virtual Assistant (Alexa, Google Assistant, Siri, DuerOS, AliGenie, Others), by Component (Hardware, Software), by Application (Smart Home, Consumer, Smart Offices, Others) and Regional Forecasts 2020-2027.
  2. Consumer Insight IBP. (2019). Mobile Telecommunications Planning Survey in the first half of 2019 (29th)
  3. Consumer Insight IBP. (2019). Mobile Telecommunications Planning Survey in the second half of 2019 (30th).
  4. Consumer Insight IBP. (2020). Mobile Telecommunications Planning Survey in the first half of 2020 (31st).
  5. Consumer Insight IBP. (2020). Mobile Telecommunications Planning Survey in the second half of 2020 (32nd).
  6. H. Shaban. (2018. 05. 25). An Amazon Echo recorded a family's conversation, then sent it to a random person in their contacts, report says. The Washington Post.
  7. D. M. Blei, A. Y. Ng & M. I. Jordan. (2013). Latent dirichlet allocation. the Journal of machine Learning research, 3, 993-1022.
  8. D. Watkins. (2019). Smart Speaker Shipment and Installed Base Forecast by Voice OS by Region: 2014 to 2024.
  9. Voicebot. (2019). Smart Speaker Consumer Adoption Report.
  10. Strategy Analytics. (2021). Strategy Analytics Smart Speaker and Screens Service.
  11. H. J. Hwang, H. R. Shim & J. H. Choi. (2016). Exploration of User Experience Research Method with Big Data Analysis : Focusing on the Online Review Analysis of Echo. The Korea Contents Society, 16(8), 517-528. DOI : 10.5392/JKCA.2016.16.08.517
  12. H. J. Lee. (2018). A Ghost in the Shell? Influences of AI Features on Product Evaluations of Smart Speakers with Customer Reviews. Journal of Information Technology Services, 17(2), 191-205 DOI : 10.9716/KITS.2018.17.2.191
  13. H. J. Lee. (2019). Customer Attitude to Artificial Intelligence Features: Exploratory Study on Customer Reviews of AI Speakers. The Knowledge Management Society of Korea, 20(2), 25-42. DOI : 10.15813/kmr.2019.20.2.002
  14. J. M. Lee, H. S. Lee & J. H. Choi. (2019). A Study on User Experience Factors of Display-Type Artificial Intelligence Speakers through Semantic Network Analysis : Focusing on Online Review Analysis of the Amazon Echo. Journal of the Convergence on Culture Technology, 5(3), 9-23. DOI : 10.17703/JCCT.2019.5.3.9
  15. J. H. Lee, H. J. Seon, & H. J. Lee. (2020). Positioning of Smart Speakers by Applying Text Mining to Consumer Reviews: Focusing on Artificial Intelligence Factors. Knowledge Management Research, 21(1), 197-210. DOI : 10.15813/kmr.2020.21.1.011
  16. M. K. Cho & B. J. Lee. (2021). Comparison of service quality of full service carriers in Korea using topic modeling: based on reviews from TripAdvisor. Journal of Hospitality and Tourism Studies, 23(1), 152-175. DOI : 10.31667/jhts.2021.2.86.152
  17. T. W. Lee. (2020). A Study on Analysis of Topic Modeling using Customer Reviews based on Sharing Economy: Focusing on Sharing Parking. Journal of the Korea Industrial Information Systems Research, 25(3), 39-51. DOI : 10.9723/jksiis,2020.25.3.039
  18. S. M. Lee & T. W. Lee. (2021). Establish Marketing Strategy Using Analysis of Local Currency App User Reviews -Focused on 'Dongbackjeon' and 'Incheoneum'. Journal of the Korea Contents Association, 21(4), 111-122. DOI : 10.5392/JKCA.2021.21.04.111
  19. B. K. Lee & C. H. Son. (2020). Improving evaluation metric of mobile application service with user review data. Journal of the Korea Academia-Industrial cooperation Society, 21(1), 380-386. DOI : 10.5762/KAIS.2020.21.1.380
  20. D. W. Hyun & S. Y. Lee. (2021). A Study of Big Data Analysis Regarding Smartphone User Satisfaction: Utilizing Sentiment Analysis Based on Social Media Data. Korean Journal of Converging Humanities, 9(1), 7-35. DOI : 10.14729/converging.k.2021.9.1.7
  21. H. S. Cho, S. A. Kang & M. H. Ryu. (2021). An Analysis of OTTService Review Using Text Mining: Focusing on the Competitive Advantage of Local Service. The Journal of Korean Institute of Communications and Information Sciences, 46(4), 722-733. https://doi.org/10.7840/kics.2021.46.4.722
  22. J. H. Park, J. Y. Oh, K. E. Min & J. J. Hwa. (2019). Customized song suggestion based on SNS sentiment analysis with Artificial Intelligent Speaker. Korea Intelligent Information Systems Society, (pp. 77). Seoul : KIISS.
  23. S. U. Yoon & M. C. Kim. (2020). Topic Modeling on Fine DUst Issues Using LDA Analysis. Journal of Energy Engineering, 29(2), 23-29. DOI : 10.5855/ENERGY.2020.29.2.023
  24. J. Devlin, M. W. Chang, K. Lee & K. Toutanova. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  25. V. Sanh, L. Debut, J. Chaumond & T. Wolf. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
  26. L. Kim. (2018, July). The Lessons of Google Glass: Aligning Key Benefits and Sociability. In International Conference on Human Interface and the Management of Information. (pp. 371-380). Springer : Cham.
  27. S. K. Jang. (2021). User Experience Research on Utilization of the Folding Screen on a Foldable Phone for Privacy Protection. Archives of Design Research, 34(2), 121-131. DOI : 10.15187/adr.2021.05.34.2.121
  28. D. H. Kim, B. A. Im & S. H. Park. (2021). It Should Be Me Who Decides What to Use.": Experiences of Using Menstrual Cups in South Korea. Journal of Korean Public Health Nursing, 35(1), 104-119. DOI : 10.5932/JKPHN.2021.35.1.104
  29. S. J. Park & S. I. Kim. (2019). A Study on the User Experience for Airline ticket Reservation Service - Focusing on Skyscanner App-. Journal of the Korea Convergence Society, 10(2), 195-200. DOI : 10.15207/JKCS.2019.10.2.195
  30. J. N. Shin & J. Y. Heo. (2021). A Case Study on Digital Transformation of Cultural Performances from a User Experience Perspective - Focusing on BTS Online Concert -. Journal of the Korean Society Design Culture, 27(1), 209-223. DOI : 10.18208/ksdc.2020.27.1.209
  31. B. Yang, Y. Liu, Y. Liang & M. Tang. (2019). Exploiting user experience from online customer reviews for product design. International Journal of Information Management, 46, 173-186. https://doi.org/10.1016/j.ijinfomgt.2018.12.006