Journal of the Korean Society for Library and Information Science (한국문헌정보학회지)

Korean Society For Library And Information Science (한국문헌정보학회)
권호 표지
DOI QR코드

DOI QR Code

Analyzing Students' Non-face-to-face Course Evaluation by Topic Modeling and Developing Deep Learning-based Classification Model

토픽 모델링 기반 비대면 강의평 분석 및 딥러닝 분류 모델 개발

  • 한지영 (연세대학교 문헌정보학과) ;
  • 허고은 (연세대학교 문헌정보학과)
  • Received : 2021.10.18
  • Accepted : 2021.11.15
  • Published : 2021.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Due to the global pandemic caused by COVID-19 in 2020, there have been major changes in the education sites. Universities have fully introduced remote learning, which was considered as an auxiliary education, and non-face-to-face classes have become commonplace, and professors and students are making great efforts to adapt to the new educational environment. In order to improve the quality of non-face-to-face lectures amid these changes, it is necessary to study the factors affecting lecture satisfaction. Therefore, This paper presents a new methodology using big data to identify the factors affecting university lecture satisfaction changed before and after COVID-19. We use Topic Modeling method to analyze lecture reviews before and after COVID-19, and identify factors affecting lecture satisfaction. Through this, we suggest the direction for university education to move forward. In addition, we can identify the factors of satisfaction and dissatisfaction of lectures from multiangle by establishing a topic classification model with an F1-score of 0.84 based on KoBERT, a deep learning language model, and further contribute to continuous qualitative improvement of lecture satisfaction.

2020년 신종 코로나바이러스 감염증(코로나19)으로 인한 전 세계적인 팬데믹으로 교육 현장에도 큰 변화가 있었다. 대학에서는 보조 교육 수단으로 생각했던 원격수업을 전면 도입하였고 비대면 수업이 일상화되어 교수자와 학생들은 새로운 교육환경에 적응하기 위해 큰 노력을 기울이고 있다. 이러한 변화 속에서 비대면 강의의 질적 향상을 위하여 강의 만족도 영향요인에 관한 연구가 필요하다. 본 연구는 코로나 전과 후로 변화된 대학 강의 만족도 영향요인을 파악하기 위해 빅데이터를 활용한 새로운 방법론을 제시하고자 한다. 토픽 모델링을 활용하여 코로나 전과 후의 강의평을 분석하고 이를 통해 강의 만족도 영향요인을 파악하여 대학교육이 나아가야 할 방향성을 제언하였다. 또한, 딥러닝 언어 모델인 KoBERT를 기반으로 0.84의 F1-score를 보이는 토픽 분류 모델을 구축함으로써 강의의 만족, 불만족 요인을 다각도로 파악할 수 있으며 이를 통해 강의 만족도의 지속적인 질적 향상에 기여할 수 있다.

Keywords

References

  1. Kang, M. S. & Park, S. K. (2011). Assessing the effects of service quality on student satisfaction, trust, commitment and loyalty: the case of university education. Academy of Customer Satisfaction Management, 13(1), 129-149.
  2. Kee, Y. H. & Roh, H. J. (2005). A comparison study on the effects of blended learning course and on-line course in a university class. The Journal of Lifelong Education and HRD, 1(1), 63-79.
  3. Kim, K. K., Kim, Y. H., & Kim, J. H. (2018). A study on customer satisfaction of mobile shopping apps using topic analysis of user reviews. The Journal of Society for e-Business Studies, 23(4), 41-62. https://doi.org/10.7838/JSEBS.2018.23.4.041
  4. Kim, D. W., Kang, J. Y., & Im, J. I. (2016). Comparative analysis of job satisfaction factors, using LDA topic modeling by industries: the case study of job planet reviews. Korea Society of IT Services, 15(3), 157-171.
  5. Kim, M. Y. & Kim, M. Y. (2020). An analysis on perception of the satisfaction with university liberal education and learning outcomes. Research on Liberal Arts Education, 14(1), 193-218.
  6. Kim, S. H. (2017). Education quality improvement by analyzing course evaluation. Research Institute for Social Science, 30(1), 147-174.
  7. Kim, S. R. (2021). Development and Validation of Graduate Student' Perceived Educational Satisfaction Model and Scale. Doctoral dissertation, Korea University.
  8. Kim, J. Y. & Park, E. H. (2017). E-learning course reviews analysis based on big data analytics. Journal of the Korean Information and Communication Society, 21(2), 423-428. https://doi.org/10.6109/jkiice.2017.21.2.423
  9. Kim, T. K., Choi, H. R., & Lee, H. C. (2016). A study on research trends in fintech using topic modeling. Journal of Korea Academia-Industrial cooperation Society(JKAIS), 17(11), 670-681. https://doi.org/10.5762/KAIS.2016.17.11.670
  10. Kim, P. J. (2019). An analytical study on automatic classification of domestic journal articles using random forest. Journal of the Korean Society for Information Management(JKOSIM), 36(2), 57-77. https://doi.org/10.3743/KOSIM.2019.36.2.057
  11. Kim, H. C. S., Ahn, D. J., Yim, J. H., & Lee, H. Y. (2017). A study on automatic classification of record text using machine learning. Journal of the Korean Society for Information Management (JKOSIM), 34(4), 321-344. https://doi.org/10.3743/KOSIM.2017.34.4.321
  12. Noh, Y. & Lee, K. K. (2020). A study on factors affecting learner satisfaction in none-face-to-face online education. Academy of Customer Satisfaction Management, 22(3), 107-126. https://doi.org/10.34183/KCSMA.22.3.5
  13. Park, E. L. & Cho, S. (2014). KoNLPy: korean natural language processing in python. Proceedings of the 26th Annual Conference on Human & Cognitive Language Technology, 6, 133-136.
  14. Park, J. H. & Song, M. (2013). A study on the research trends in library & information science in korea using topic modeling. Journal of the Korean Society for Information Management (JKOSIM), 30(1), 7-32. https://doi.org/10.3743/KOSIM.2013.30.1.007
  15. Park, J. D. (2015). A study on mapping users' topic interest for question routing for community-based Q&A service. Journal of the Korean Society for Information Management (JKOSIM), 32(3), 397-412. https://doi.org/10.3743/KOSIM.2015.32.3.397
  16. Park, J. S., Hong, S. G., & Kim, J. W. (2017). A study on science technology trend and prediction using topic modeling. Journal of the Korea Industrial Information Systems Research, 22(4), 19-28. https://doi.org/10.9723/JKSIIS.2017.22.4.019
  17. Baek, S. H. (2021). Verification of predictive factors for gender equality and threat from elder. The Journal of Humanities and Social Science, 12(2), 2543-2556.
  18. Suh, K. W. (2011). Factors affecting online lecture satisfaction and intention to recommend. Journal of Cyber Education, 5(2), 159-178.
  19. Song, Y. H. (2020). The influence of instructor's non-verbal communication on college student's emotional presence, rapport and learning satisfaction. The Korea Contents Society, 20(10), 259-267.
  20. Song, Y. H. & Ji, S. G. (2012). Relationships among instructor sense of humor, rapport, flow, and satisfaction in university classes. Journal of Educational Studies, 43(4), 245-269.
  21. Sin, J. Y., Kwon, L. M., & Moon, S. H. (2009). AHP analysis for the factors that influence college student's satisfaction on the business courses. Korean Business Education Review, 55(1), 53-73.
  22. Sim, J. K. (2021). A study on automatic classification of profanity sentences of elementary school students using BERT. Journal of Creative Information Culture, 7(2), 91-98. https://doi.org/10.32823/JCIC.7.2.202105.91
  23. Yuk, J. H. & Song, M. (2018). A study of research on methods of automated biomedical document classification using topic modeling and deep learning. Journal of the Korean Society for Information Management(JKOSIM), 35(2), 63-88. https://doi.org/10.3743/KOSIM.2018.35.2.063
  24. Lee, M. S. (2020). Improvement of Recommendation System Using Attribute-based Opinion Mining of Online Customer Reviews. Master's thesis, Kookmin University.
  25. Lee, S. B., Kim, S. D., Lee, J. H., Ko, Y. S., & Song, M. (2021). Building and analyzing panic disorder social media corpus for automatic deep learning classification model. Journal of the Korean Society for Information Management(JKOSIM), 38(2), 153-172. https://doi.org/10.3743/KOSIM.2021.38.2.153
  26. Lee, S. K., Choi, S. B., & Kim, H. W. (2019). An Exploratory Study of E-learning satisfaction: a mixed methods of text mining and interview approaches, Information Systems Review, 21(1), 39-59. https://doi.org/10.14329/isr.2019.21.1.039
  27. Jo, M. W. & Kim, J. Y. (2021). A study on the perception and the satisfaction of online classes at K university in the non-face-to-face era. The Journal of Humanities and Social Science, 12(2), 1399-1414.
  28. Jin, S. A., Heo, G. E., Jeon, Y. K., & Song, M. (2013). Topic-network based topic shift detection on twitter. Journal of the Korean Society for Information Management(JKOSIM), 30(1), 285-302. https://doi.org/10.3743/KOSIM.2013.30.1.285
  29. Choi, K. H. & Kang. S. (2011). A study measuring university educational service quality using importance-satisfaction transformed index. Journal of the Korean Data & Information Science Society, 22(4), 765-773.
  30. Han, E. S. & Kim, J. D. (2003). An analysis of influential factors on college of education students' academic satisfaction. The Korean Society for the Study of Teacher Education, 20(3), 313-335.
  31. Ham, E. H., Park, S. O., & Kim, E. K. (2017) Using subscale scores of university student satisfaction survey: an application of bifactor models. Asian Journal of Education, 18(4), 713-738. https://doi.org/10.15753/aje.2017.12.18.4.713
  32. Hwang, S. H. & Kim, D. H. (2020). BERT-based classification model for korean documents. The Journal of Society for e-Business Studies, 25(1), 203-214. https://doi.org/10.7838/JSEBS.2020.25.1.203
  33. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. The Journal of Machine Learning Research, 3, 993-1022.
  34. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  35. Hofmann, T. (1999). Probabilistic latent semantic analysis. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, 289-296.
  36. Jeon, Heewon (2021). KoBERT. Available: https://github.com/SKTBrain/KoBERT
  37. Kendon, A. (1981). Nonverbal Communication: Interaction and Gesture. The Hague: Mouton Publisher.
  38. Mimno, D. & McCallum, A. (2012). Topic models conditioned on arbitrary features with dirichlet-multinomial regression. arXiv preprint arXiv:1206.3278.
  39. Rosen-Zvi, M., Griffiths, T., Steyvers, M., & Smyth, P. (2012). The author-topic model for authors and documents. arXiv preprint arXiv:1207.4169.