Journal of Information Technology Services (한국IT서비스학회지)

Korea Society of IT Services (한국IT서비스학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Improving the Quality of Synchronous and Asynchronous Online Learning Systems: Using IPA and Topic Modeling

실시간 및 비실시간 온라인 학습 시스템의 품질 개선 연구: IPA와 토픽 모델링을 활용하여

  • 황기석 (한양대학교 일반대학원 경영학과) ;
  • 차경진 (한양대학교 경영학부)
  • Received : 2024.12.17
  • Accepted : 2025.01.08
  • Published : 2025.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

This study aims to propose quality improvement measures for online learning systems by comparing learning experiences in synchronous and asynchronous learning modes. Based on 1,551 learner reviews, topic modeling identifies key learning attributes and positive and negative importance-performance analysis (IPA) is performed. A t-test is also performed to quantitatively compare 'instructor delivery skills' and overall satisfaction. The results show that 'instructor delivery skills' is the most important key attribute in both learning modes, and 'instructor delivery skills' and overall satisfaction are rated significantly higher in the asynchronous learning mode than in the synchronous learning mode. In synchronous learning mode, 'content quality' and 'learning autonomy' are identified as key attributes with ambivalence. In the asynchronous learning mode, 'content quality' serves as the main attribute that leads to learner satisfaction, while 'learning autonomy' functions as a stable attribute that does not lead to dissatisfaction. In particular, this study is significant in that it systematically identifies the impact of key attributes of each learning mode on the learning experience using positive and negative IPA. The findings suggest the need to mitigate the negative impact of time constraints on the learning experience in synchronous learning mode and complement interactivity in asynchronous learning mode. It also provides practical suggestions for improving the quality of the learning experience through customized improvement strategies for each learning mode.

Keywords

References

  1. 권영선, 최혁준, "공공 평생교육기관의 e-러닝교육 서비스품질 향상에 관한 연구", e-비즈니스연구, 제22권, 제4호, 2021, 55-69.
  2. 김소연, 김창희, 박한승, 김채원, 오윤정, 최진석, "COVID-19 이후 대학 E-learning 만족도 요인 분석: IPA 분석을 중심으로", 글로벌경영학회지, 제18권, 제3호, 2021, 133-154.
  3. 김소희, 김무건, 류민호, "국내 모바일 뱅킹 애플리케이션에 대한 이용자 중요도-만족도 분석 (IPA): 구글 플레이스토어 리뷰 데이터를 활용하여", 한국산업정보학회논문지, 제27권, 제6호, 2022, 115-126.
  4. 나소정, "전통적 IPA 와 수정된 IPA 를 이용한 항공서비스학과의 온라인 교육서비스 품질에 관한 연구", 관광경영연구, 제25권, 제2호, 2021, 167-182.
  5. 선영지, 이중정, 윤혜정, "AI 비서 서비스의 중요도와 만족도 분석 연구", 한국IT서비스학회지, 제20권, 제4호, 2021, 81-93.
  6. 오민재, 류재숙, "전통적 IPA와 수정 IPA의 비교: 인천차이나타운의 관광매력성 평가", 관광연구저널, 제30권, 제7호, 2016, 129-142.
  7. 이명성, 김한성, "온라인 서점의 서비스 품질 요인에 따른 소비자 중요도 및 만족도 분석", 한국IT서비스학회지, 제20권, 제3호, 2021, 103-118.
  8. 황기석, 차경진, "비대면 채널 유형에 따른 이용자 만족도 비교 분석 연구", 스마트미디어저널, 제13권, 제7호, 2024, 27-35.
  9. Ali, R. and E.M. Leeds, "The impact of faceto-face orientation on online retention: A pilot study", Online Journal of Distance Learning Administration, Vol.12, No.4, 2009.
  10. Amiti, F., "Synchronous and asynchronous Elearning", European Journal of Open Education and E-Learning Studies, Vol.5, No.2, 2020.
  11. Azzopardi, E. and R. Nash, "A critical evaluation of importance-performance analysis", Tourism Management, Vol.35, 2013, 222-233.
  12. Ban, O.I., R. Hatos, N.G. Bugnar, D. Sasu, A.L. Popa, and A.F. Fora, "Evaluation of the Quality of Higher Education Services by Revised IPA in the Perspective of Digitization", Sustainability, Vol.16, No.7, 2024, 3017. B
  13. Bernard, R.M., P.C. Abrami, E. Borokhovski, C.A. Wade, R.M. Tamim, M.A. Surkes, and E.C. Bethel, "A meta-analysis of three types of interaction treatments in distance education", Review of Educational Research, Vol.79, No.3, 2009, 1243-1289.
  14. Bi, J.W., Y. Liu, Z.P. Fan, and J. Zhang, "Wisdom of crowds: Conducting importance-performance analysis (IPA) through online reviews", Tourism Management, Vol.70, 2019, 460-478.
  15. Bismala, L. and Y.H. Manurung, "Student Satisfaction in E-Learning along the COVID-19 Pandemic with Importance Performance Analysis", International Journal of Evaluation and Research in Education, Vol.10, No.3, 2021, 753-759.
  16. Blei, D.M., "Probabilistic topic models", Communications of the ACM, Vol.55, No.4, 2012, 77-84.
  17. Blei, D.M., A.Y. Ng, and M.I. Jordan, "Latent dirichlet allocation", Journal of Machine Learning Research, Vol.3, 2003, 993-1022.
  18. Blei, D. and J. Lafferty, "Correlated topic models", Advances in Neural Information Processing Systems, Vol.18, 2006, 147.
  19. Boley, B.B., N.G. McGehee, and A.T. Hammett, "Importance-performance analysis (IPA) of sustainable tourism initiatives: The resident perspective", Tourism Management, Vol.58, 2017, 66-77.
  20. Caber, M., T. Albayrak, and K. Matzler, "Classification of the destination attributes in the content of competitiveness (by revised importance-performance analysis)", Journal of Vacation Marketing, Vol.18, No.1, 2012, 43-56.
  21. Chen, Y., Y. Zhong, S. Yu, Y. Xiao, and S. Chen, "Exploring bidirectional performance of hotel attributes through online reviews based on sentiment analysis and Kano-IPA model", Applied Sciences, Vol.12, No.2, 2022, 692.
  22. Choudhury, S. and S. Pattnaik, "Emerging themes in e-learning: A review from the stakeholders' perspective", Computers & Education, Vol.144, 2020, 103657.
  23. Cladera, M., "An application of importance-performance analysis to students' evaluation of teaching", Educational Assessment, Evaluation and Accountability, Vol.33, No.4, 2021, 701-715.
  24. Coman, C., L.G. Țîru, L. Meseșan-Schmitz, C. Stanciu, and M.C. Bularca, "Online teaching and learning in higher education during the coronavirus pandemic: Students' perspective", Sustainability, Vol.12, No.24, 2020, 10367.
  25. Engelbrecht, E., "Adapting to changing expectations: Post-graduate students' experience of an e-learning tax program", Computers & Education, Vol.45, No.2, 2005, 217-229.
  26. Esmailpour, J., K. Aghabayk, M.A. Vajari, and C. De Gruyter, "Importance–Performance Analysis (IPA) of bus service attributes: A case study in a developing country", Transportation research part a: policy and practice, Vol.142, 2020, 129-150.
  27. Groves, R.M., "Nonresponse rates and nonresponse bias in household surveys", International Journal of Public Opinion Quarterly, Vol.70, No.5, 2006, 646-675.
  28. Guo, Y., S.J. Barnes, and Q. Jia, "Mining meaning from online ratings and reviews: Tourist satisfaction analysis using latent dirichlet allocation", Tourism Management, Vol.59, 2017, 467-483.
  29. Jung, Y. and Y. Suh, "Mining the voice of employees: A text mining approach to identifying and analyzing job satisfaction factors from online employee reviews", Decision Support Systems, Vol.123, 2019, 113074.
  30. Lakhiwal, A., H. Bala, and P.M. Léger, "Ambivalence is better than indifference: behavioral and neurophysiological assessment of ambivalence in online environments", MIS Quarterly, Vol.47, No.2, 2022, 705-732.
  31. Martilla, J.A. and J.C. James, "Importance-performance analysis", Journal of Marketing, Vol.41, No.1, 1977, 77-79.
  32. Mejia, C., M. Bąk, P. Zientara, and M. Orlowski, "Importance-performance analysis of socially sustainable practices in US restaurants: A consumer perspective in the quasi-post-pandemic context", International Journal of Hospitality Management, Vol.103, 2022, 103209.
  33. Nam, S. and H. C. Lee, "A text analytics-based importance performance analysis and its application to airline service", Sustainability, Vol.11, No.21, 2019, 6153.
  34. Oh, H., "Revisiting importance–performance analysis", Tourism Management, Vol.22, No.6, 2001, 617-627.
  35. Qi, J., Z. Zhang, S. Jeon, and Y. Zhou, "Mining customer requirements from online reviews: A product improvement perspective", Information & Management, Vol.53, No.8, 2016, 951-963.
  36. Rahman, M.N.A., S.N.A.S. Zamri, and L.K. Eu, "A meta-analysis study of satisfaction and continuance intention to use educational technology", International Journal of Academic Research in Business and Social Sciences, Vol.7, No.4, 2017, 1059-1072.
  37. Sever, I, "Importance-performance analysis: A valid management tool?", Tourism Management, Vol.48, 2015, 43-53.
  38. Shoepe, T.C., J.F. McManus, S.E. August, N.L. Mattos, T.C. Vollucci, and P.R. Sparks, "Instructor prompts and student engagement in synchronous online nutrition classes", American Journal of Distance Education, Vol.34, No.3, 2020, 194-210.
  39. Sörensson, A. and Y. von Friedrichs, "An importance–performance analysis of sustainable tourism: A comparison between international and national tourists", Journal of Destination Marketing & Management, Vol.2, No.1, 2013, 14-21.
  40. Sumrit, D. and K. Sowijit, "Winning customer satisfaction toward omnichannel logistics service quality based on an integrated importance-performance analysis and three factor theory: Insight from Thailand", Asia Pacific Management Review, Vol.28, No.4, 2023, 531-543.
  41. Tirunillai, S. and G.J. Tellis, "Mining marketing meaning from online chatter: Strategic brand analysis of big data using latent dirichlet allocation", Journal of Marketing Research, Vol.51, No.4, 2014, 463-479.
  42. Wu, J. and T. Yang, "Service attributes for sustainable rural tourism from online comments: Tourist satisfaction perspective", Journal of Destination Marketing & Management, Vol.30, 2023, 100822.
  43. Wu, J. and N. Zhao, "What consumer complaints should hoteliers prioritize? Analysis of online reviews under different market segments", Journal of Hospitality Marketing & Management, Vol.32, No.1, 2023, 1-28.
  44. Yunusa, A.A. and I.N. Umar, "A scoping review of critical predictive factors (CPFs) of satisfaction and perceived learning outcomes in E-learning environments", Education and Information Technologies, Vol.26, No.1, 2021, 1223-1270.
  45. Zhang, Z., Z. Zhang, and Y. Yang, "The power of expert identity: How website-recognized expert reviews influence travelers' online rating behavior", Tourism Management, Vol.55, 2016, 15-24.