Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Toward understanding learning patterns in an open online learning platform using process mining

프로세스 마이닝을 활용한 온라인 교육 오픈 플랫폼 내 학습 패턴 분석 방법 개발

  • Taeyoung Kim (School of Information Convergence, Kwangwoon University) ;
  • Hyomin Kim (School of Information Convergence, Kwangwoon University) ;
  • Minsu Cho (School of Information Convergence, Kwangwoon University)
  • 김태영 (광운대학교 정보융합학부) ;
  • 김효민 (광운대학교 정보융합학부) ;
  • 조민수 (광운대학교 정보융합학부)
  • Received : 2023.06.12
  • Accepted : 2023.06.22
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Due to the increasing demand and importance of non-face-to-face education, open online learning platforms are getting interests both domestically and internationally. These platforms exhibit different characteristics from online courses by universities and other educational institutions. In particular, students engaged in these platforms can receive more learner autonomy, and the development of tools to assist learning is required. From the past, researchers have attempted to utilize process mining to understand realistic study behaviors and derive learning patterns. However, it has a deficiency to employ it to the open online learning platforms. Moreover, existing research has primarily focused on the process model perspective, including process model discovery, but lacks a method for the process pattern and instance perspectives. In this study, we propose a method to identify learning patterns within an open online learning platform using process mining techniques. To achieve this, we suggest three different viewpoints, e.g., model-level, variant-level, and instance-level, to comprehend the learning patterns, and various techniques are employed, such as process discovery, conformance checking, autoencoder-based clustering, and predictive approaches. To validate this method, we collected a learning log of machine learning-related courses on a domestic open education platform. The results unveiled a spaghetti-like process model that can be differentiated into a standard learning pattern and three abnormal patterns. Furthermore, as a result of deriving a pattern classification model, our model achieved a high accuracy of 0.86 when predicting the pattern of instances based on the initial 30% of the entire flow. This study contributes to systematically analyze learners' patterns using process mining.

비대면 교육의 중요성 및 필요에 따른 수요가 증가함에 따라 국내외 온라인 교육 오픈 플랫폼이 활성화되고 있다. 본 플랫폼은 대학 등 교육 전문기관과 달리 학습자의 자율성이 높은 특징을 가지며 이에 따라 개인화된 학습 도구를 지원하기 위한 학습 행동 데이터의 분석 연구가 중요시 되고 있다. 실제적인 학습 행동을 이해하고 패턴을 도출하기 위하여 프로세스 마이닝이 다수 활용되었지만 온라인 교육 플랫폼과 같이 자기 관리형(Self-regulated) 환경에서의 학습 로그를 기반한 사례는 부족하다. 또한, 대부분 프로세스 모델 도출 등의 모델 관점에서의 접근이며 분석 결과의 실제적인 적용을 위한 개별 패턴 및 인스턴스 관점에서의 방법 제시는 미흡하다. 본 연구에서는 온라인 교육 오픈 플랫폼 내 학습 패턴을 파악하기 위하여 프로세스 마이닝을 활용한 분석 방법을 제시한다. 학습 패턴을 다각도로 분석하기 위하여 모델, 패턴, 인스턴스 관점에서의 분석 방법을 제시하며, 프로세스 모델 발견, 적합도 검사, 군집화 기법, 예측 알고리즘 등 다양한 기법을 활용한다. 본 방법은 국내 오픈 교육 플랫폼 내 기계학습 관련 강좌의 학습 로그를 추출하여 분석하였다. 분석 결과 온라인 강의의 특성에 맞게 비구조화된 프로세스 모델을 도출할 수 있었으며 구체적으로 한 개의 표준 학습 패턴과 세 개의 이상 학습 패턴으로 세분화할 수 있었다. 또한, 인스턴스별 패턴 분류 예측 모델을 도출한 결과 전체 흐름 중 초기 30%의 흐름을 바탕으로 예측하였을 때 0.86의 분류 정확도를 보였다. 본 연구는 프로세스 마이닝을 활용하여 학습자의 패턴을 체계적으로 분석한다는 점에서 기여점을 가진다.

Keywords

Acknowledgement

본 연구는 네이버 커넥트재단 지원 및 한국연구재단의 지원을 받아 수행된 연구임 (NRF-2021R1G1A1094019).

References

  1. 권영옥. (2013). 빅데이터를 활용한 맞춤형 교육 서비스 활성화 방안연구. 지능정보연구, 19(2), 87-100. https://doi.org/10.13088/JIIS.2013.19.2.087
  2. 정준희. (2012). 통계적 접근법을 기초로 하는 지능형 교육 지원 시스템. 지능정보연구, 18(1), 109-123. https://doi.org/10.13088/JIIS.2012.18.1.109
  3. 강영식, 정진우, 심선영. (2023). RPA 로그 마이닝 기반 프로세스 자동화 현황 분석 - 중소기업 대상 실증 연구. 지능정보연구, 29(1), 265-288. https://doi.org/10.13088/JIIS.2023.29.1.265
  4. 김혜경, 배성아. (2019). Felder-Silverman 학습양식에 따른 영어 튜터링 프로그램의 튜터와 튜티의 학습성취도와 만족도 간의 관계 연구. 교양교육연구, 3(2), 115-140.
  5. 손세창, 윤한영. (2022). 포스트 코로나19 시대 공항의 디지털 트랜스포메이션을 통한 공항 운영에서 혁신적 비즈니스 모델 적용 연구. 한국산학기술학회논문지, 23(4), 483-492.
  6. 허묘연. (2020). 언택트 교육의 시대: 온라인 교육을 위한 준비. 차세대컨버전스정보서비스기술논문지, 9(3), 315-325.
  7. Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439-1459.
  8. Graf, S., Viola, S.R. & Leo, T. (2007). In-Depth Analysis of the Felder-Silverman Learning Style Dimensions Journal of Research on Technology in Education, 40(1), 79-93. https://doi.org/10.1080/15391523.2007.10782498
  9. Bogarin, A., Cerezo, R. & Romero, C. (2017) A survey on educational process mining WIREs Data Mining and Knowledge Discovery, 8(1). https://doi.org/10.1002/widm.1230
  10. Siemens, G. (2013). Learning Analytics: The Emergence of a Discipline American Behavioral Scientist, 57(10), 1380-1400 https://doi.org/10.1177/0002764213498851
  11. Gasevic, D., Dawson, S., Rogers, T. & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success Internet and Higher Education, 68-84. https://doi.org/10.1016/j.iheduc.2015.10.002
  12. Mukala, P., Buijs, J., Leemans, M. & Van Der Aalst, W. (2015). Exploring Students' Learning Behaviour in MOOCs Using Process Mining. BPM Reports,1510, Eindhoven University of Technology, Eindhoven, The Netherlands. https://research.tue.nl/nl/publications/exploring-students-learning-behaviour-in-moocs-using-process-mini
  13. Hachicha, W., Ghorbel, L., Champagnat, R., Zayani, C.A. & Amous, I. (2021). Using Process Mining for Learning Resource Recommendation: A Moodle Case Study. Procedia Computer Science, 853-862. https://doi.org/10.1016/j.procs.2021.08.088
  14. Schoor, C. & Bannert, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining Computers in Human Behavior, 1321-1331. https://doi.org/10.1016/j.chb.2012.02.016
  15. Juhanak, L., Zounek, J. & Rohlikova, L. (2019). Using process mining to analyze students' quiz-taking behavior patterns in a learning management system Computers in Human Behavior, 496-506. https://doi.org/10.1016/j.chb.2017.12.015
  16. Van Der Aalst, W. (2016). Process mining: data science in action (Vol. 2). Heidelberg: Springer.
  17. Augusto, A., Conforti, R., Dumas, M., La Rosa, M., Maggi, F. M., Marrella, A., ... & Soo, A. (2018). Automated discovery of process models from event logs: review and benchmark. IEEE transactions on knowledge and data engineering, 31(4), 686-705.
  18. Cho, M., Song, M., Comuzzi, M., & Yoo, S. (2017). Evaluating the effect of best practices for business process redesign: An evidence-based approach based on process mining techniques. Decision Support Systems, 104, 92-103. https://doi.org/10.1016/j.dss.2017.10.004
  19. Cardoso, J., Mendling, J., Neumann, G., & Reijers, H. A. (2006, September). A discourse on complexity of process models. In Business process management workshops (Vol. 4103, pp. 117-128).
  20. McCoy, C. & Shih P. C. (2016). Teachers as Producers of Data Analytics: A Case Study of a Teacher Focused Educational Data Science Program, Journal of Learning Analytics, 3(3), 193-214. https://doi.org/10.18608/jla.2016.33.10
  21. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11(2), 65-70. https://doi.org/10.1016/j.iheduc.2008.03.002
  22. Muslim, A., Chatti, M. A., & Guesmi, M. (2020). Open learning analytics: a systematic literature review and future perspectives. Artificial Intelligence Supported Educational Technologies, 3-29.
  23. Wang, Y., Mu, C., Li, X., & Yang, Y. (2019, August). An Applied Research on Big Data Analysis and Mining Technology in Education. In 2019 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC) (pp. 116-120). IEEE.
  24. Chen, L., Lu, M., Goda, Y., & Yamada, M. (2019). Design of Learning Analytics Dashboard Supporting Metacognition. International Association for Development of the Information Society.
  25. Fleur, D. S., van den Bos, W., & Bredeweg, B. (2020). Learning analytics dashboard for motivation and performance. In Intelligent Tutoring Systems: 16th International Conference, ITS 2020, Athens, Greece, June 8-12, 2020, Proceedings 16 (pp. 411-419). Springer International Publishing.
  26. Baneres, D., Rodriguez-Gonzalez, M. E., & Serra, M. (2019). An early feedback prediction system for learners at-risk within a first-year higher education course. IEEE Transactions on Learning Technologies, 12(2), 249-263. https://doi.org/10.1109/TLT.2019.2912167
  27. Han, J., Kim, K. H., Rhee, W., & Cho, Y. H. (2021). Learning analytics dashboards for adaptive support in face-to-face collaborative argumentation. Computers & Education, 163, 104041.
  28. Kokoc, M., & Altun, A. (2021). Effects of learner interaction with learning dashboards on academic performance in an e-learning environment. Behaviour & Information Technology, 40(2), 161-175. https://doi.org/10.1080/0144929X.2019.1680731
  29. Khalil, M., & Ebner, M. (2017). Clustering patterns of engagement in Massive Open Online Courses (MOOCs): the use of learning analytics to reveal student categories. Journal of computing in higher education, 29, 114-132. https://doi.org/10.1007/s12528-016-9126-9
  30. Viberg, O., Hatakka, M., Balter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in human behavior, 89, 98-110. https://doi.org/10.1016/j.chb.2018.07.027
  31. Trcka, N., & Pechenizkiy, M. (2009, November). From local patterns to global models: Towards domain driven educational process mining. In 2009 Ninth international conference on intelligent systems design and applications (pp. 1114-1119). IEEE.
  32. Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students' self-regulated learning. Metacognition and learning, 9, 161-185. https://doi.org/10.1007/s11409-013-9107-6
  33. Cairns, A. H., Gueni, B., Fhima, M., Cairns, A., David, S., & Khelifa, N. (2014, July). Towards custom-designed professional training contents and curriculums through educational process mining. In The fourth international conference on advances in information mining and management (pp. 53-58).