Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Generative AI-Based Feedback Techniques for Tutoring Beginners' Error Codes on Online Judge Platforms

  • Juyeon Lee (Dept. of Computer Education, Korea National University of Education) ;
  • Seung-Hyun Kim (Dept. of Computer Education, Korea National University of Education)
  • Received : 2024.06.03
  • Accepted : 2024.07.26
  • Published : 2024.08.30
Download PDF
⟨ Previous Next ⟩

Abstract

The rapid advancement of computer technology and artificial intelligence has significantly impacted software education in Korea. Consequently, the 2022 revised curriculum demands personalized education. However, implementing personalized education in schools is challenging. This study aims to facilitate personalized education by utilizing incorrect codes and error information submitted by beginners to construct prompts. And the difference in the frequency of correct feedback generated by the generative AI model and the prompts was examined. The results indicated that providing appropriate error information in the prompts yields better performance than relying solely on the excellence of the generative AI model itself. Through this research, we hope to establish a foundation for the realization of personalized education in programming education in Korea.

컴퓨터 기술과 인공지능의 비약적인 발전이 국내 소프트웨어 교육에서도 많은 영향을 끼치고 있다. 이에 따라 2022 개정 교육과정에서도 맞춤형 교육을 요구하게 되었지만, 학교에서 맞춤형 교육을 실현하기에는 어려움이 있다. 이에 본 연구에서는 맞춤형 교육 실현을 위해 초보 학습자가 제출한 오답 코드와 오답 정보들을 활용하여 적절한 피드백 생성을 위한 프롬프트를 구성하였다. 그리고 생성형 인공지능 모델과 프롬프트 조합에 따른 정상 피드백 생성 빈도의 차이를 실제 데이터를 활용하여 분석하였다. 그 결과, 생성형 인공지능 모델 자체의 우수성보다 오답 정보를 포함한 프롬프트가 더 우수한 피드백 생성 성능을 나타내는 것을 확인하였다. 본 연구를 통해 국내 프로그래밍 교육에서 맞춤형 교육의 실현을 위한 토대가 되기를 기대한다.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. RS-2023-00211436).

References

  1. S. C. Kang, S. J. Ahn, Y. H. Sung, Y. S. Jeong, K. Y. A, J. H. Seo and S. Y. Park, "Empirical Data analysis Report On Overseas Software Education Current Status, "Korea Education and Research Information Service. 
  2. MINISTRY OF EDUCATION, "the 2022 revised curriculum, " 2022. 
  3. MINISTRY OF EDUCATION, "Digital education vision declaration ceremony and academic conference (conference), press realease, "2022. 
  4. MINISTRY OF EDUCATION, "Digital-based education innovation plan, "2023. 
  5. B. S. Bloom, "The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring," Educational researcher, 13(6), pp. 4-16 1984. 
  6. Korean Educational Development Institute, "STATISTICAL YEARBOOK OF EDUCATION, "2023. 
  7. S. Kim and Y. Jang, "An Analysis of Factors Influencing High School Students' intention to continue using AI chatbots in Programming Education," The Journal of Korean association of computer education, 26(5), pp. 93-105 2023. DOI:10.32431/kace.2023.26.5.008. 
  8. Kim Myunghee, Han Jiwon and Yoo Yung-eui, "A Study on the Effects and Participant Perception of Classes Applying Artificial Intelligence-Based Personalized Learning," Journal of Education & Culture, 29(1) 2023. DOI:10.24159/joec.2023.29.1.137. 
  9. Do Jaewoo, Jeongin Eur, Na Yong Jae and Sujin Kim, "A Study of Teachers' Use and Perception of Learning Analytics based Dashboard for Customized Education," The Journal of Korean Teacher Education, 39(4), pp. 261-289. DOI:10.24211/tjkte.2022.39.4.261. 
  10. Soohwan Lee and Song Kisang, "Exploration of Domestic Research Trends on Educational Utilization of Generative Artificial Intelligence," The Journal of Korean association of computer education, 26(6), pp. 15-27 2023. DOI:10.32431/kace.2023.26.6.002. 
  11. A. Kurnia, A. Lim and B. Cheang, "Online Judge," Computers & Education, 36(4) 2001. 
  12. X. Du, C. Yi, Y. Wei, S. Feng and Z. Gong, "Design of Automata Online Judge," In 2010 2nd International Conference on Information Engineering and Computer Science, pp. 1-4 2010. DOI: 10.1109/ICIECS.2010.5677856 
  13. Baekjoon Online Judge, https://www.acmicpc.net/, 2024(Apr 22,). 
  14. Jungol, https://www.jungol.co.kr/, 2024(Apr 22,). 
  15. W. Chang and S. Kim, "Development and application of algorithm judging system : analysis of effects on programming learning," The Journal of Korean association of computer education, 17(4), pp. 45-57 2014. DOI:http://dx.doi.org/10.32431/kace.2014.17.4.005. 
  16. J. Shim and J. M. Chae, "Development of On-line Judge System based onBlock Programming Environment," The Journal of Korean association of computer education, 21(6), pp. 1-10 2018. DOI : 10.32431/kace.2018.21.4.001 
  17. S. Jung, "Design of Block Coding Online Judge System," Journal of the Edutainment, 2(1), pp. 57-71 2020. DOI:10.36237/koedus.2.1.57. 
  18. E. Sohn and J. Kim, "Implementation of an Algorithmic Trading Problem Evaluation System for Online Programming Courses," KTCP, 29(11) 2023. DOI:10.5626/ktcp.2023.29.11.525. 
  19. S. Kim, S. Oh and S. Jeong, "Development and Application of Problem Bank of Problem Solving Programming Using Online Judge System in Data Structure Education," , 21(6), pp. 11-20 2018. DOI : 10.32431/kace.2018.21.4.002 
  20. H. Go, J. H. Jeon and Y. Lee, "A Study on the Development of Problem Bank for Programming.Math Convergence Education in Programming Automatic Assessment System, ", 27(2)2023. DOI:10.14352/jkaie.2023.27.2.141. 
  21. K. Hur, "Validity Analysis of Python Automatic Scoring Exercise-Problems using Machine Learning Models," , 15(1), pp. 193-198 2023. DOI:10.14702/JPEE.2023.193. 
  22. H. W. Kim, H. J. Yun and K. Kim, "A Study on the Intelligent Online Judging System Using User-Based Collaborative Filtering," , 29(1), pp. 273-285 2024. DOI:10.9708/jksci.2024.29.01.273. 
  23. H. Mun, S. Kim, J. Kim and Y. Lee, "A Source-code Similarity-based Automatic Tutoring Method for Online Coding Test Service," JOK, 48(9), pp. 1044-1051 2021. DOI:10.5626/jok.2021.48.9.1044. 
  24. T. Crow, A. Luxton-Reilly and B. Wuensche, "Intelligent tutoring systems for programming education: a systematic review," Proceedings of the 20th Australasian Computing Education Conference, pp. 53-62 2018. DOI:10.1145/3160489.3160492. 
  25. JR Anderson and BJ Reiser, "The LISP Tutor," , 10(4), pp. 159-175 1985. 
  26. S. Choi, D. Lee, J. Kim, Y. Jang and H. Kim, "Designing LLM-based Code Reviewing Learning Environment for Programming Education," , 26(5), pp. 1-11 2023. DOI:10.32431/kace.2023.26.5.001. 
  27. S. Kim, "Developing Code Generation Prompts for Programming Education with Generative AI," , 26(5) 2023. DOI:10.32431/kace.2023.26.5.009. 
  28. S. Lee and K. Song, "Prompt engineering to improve the performance of teaching and learning materials Recommendation of Generative Artificial Intelligence," , 28(8) 2023. DOI:10.9708/jksci.2023.28.08.195. 
  29. Code Generation on HumanEval, https://paperswithcode.com/sota/code-generation-on-humaneval , .05.16 2024. 
  30. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. Ponde De Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. Mcgrew, D. Amodei, S. Mccandlish, I. Sutskever and W. Zaremba, "Evaluating Large Language Models Trained on Code," arXiv preprint arXiv:2107.03374, 2021. 
  31. Qingdao Online Judge, https://github.com/QingdaoU/OnlineJudge.