Journal of Engineering Education Research (공학교육연구)

Korean Society for Engineering Education (한국공학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of ChatGPT's Coding Capabilities in Foundational Programming Courses

기초 프로그래밍 과목에서의 ChatGPT의 코딩 역량 분석

  • Nah, Jae-Ho (Department of Computer Science, Sangmyung University)
  • 나재호 (상명대학교 컴퓨터과학과)
  • Received : 2023.10.12
  • Accepted : 2023.11.23
  • Published : 2023.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

ChatGPT significantly broadens the application of artificial intelligence (AI) services across various domains, with one of its primary functions being assistance in programming and coding. Nevertheless, due to the short history of ChatGPT, there have been few studies analyzing its coding capabilities in Korean higher education. In this paper, we evaluate it using exam questions from three foundational programming courses at S University. According to the experimental results, ChatGPT successfully generated Python, C, and JAVA programs, and the code quality is on par with that of high-achieving students. The powerful coding capabilities of ChatGPT imply the need for a strict prohibition of its usage in coding tests; however, it also suggests significant potential for enhancing practical exercises in the educational aspect.

Keywords

References

  1. 나재호(2023). 컴퓨터과학 분야에서의 비대면 공학설계입문 강의 사례 연구. 공학교육연구, 26(1), 12-19. https://doi.org/10.18108/JEER.2023.26.1.12
  2. 정한별.한경희(2023). ChatGPT가 한국 공학교육에 던지는 질문:그 의미와 과제. 공학교육연구, 26(5), 17-28. https://doi.org/10.18108/JEER.2023.26.5.17
  3. 한국전력공사(2023). 주택용전력 요금 누진제. https://cyber.kepco.co.kr/ckepco/front/jsp/CY/H/C/CYHCHP00206.jsp
  4. Arefin, S. E. et al.(2023). Unmasking the giant: A comprehensive evaluation of ChatGPT's proficiency in coding algorithms and data structures. arXiv preprint arXiv:2307.05360.
  5. Chen, M. et al.(2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  6. Geng, C. et al.(2023). Can ChatGPT Pass An Introductory Level Functional Language Programming Course?. arXiv preprint arXiv:2305.02230.
  7. Hendrycks, D. et al.(2020). Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations (ICLR) 2020.
  8. Hu, Y. et al.(2019). Re-factoring based program repair applied to programming assignments. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), 388-398.
  9. LeetCode(2023). LeetCode: The World's Leading Online Programming. https://leetcode.com
  10. Li, K. et al.(2023). Discriminating Human-authored from ChatGPT-Generated Code Via Discernable Feature Analysis. In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), 120-127.
  11. Nijkamp, E. et al.(2023). CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. In International Conference on Learning Representations (ICLR) 2023.
  12. OpenAI(2023). Introducing ChatGPT. https://openai.com/blog/chatgpt
  13. OpenAI(2023). GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.
  14. Ouyang, L. et al.(2022). Training Language Models to Follow Instructions with Human Feedback. In Advances in Neural Information Processing Systems (NeurIPS), 35, 27730-27744.
  15. Tian, H. et al.(2023). Is ChatGPT the Ultimate Programming Assistant--How far is it?. arXiv preprint arXiv:2304.11938.
  16. Tong, A.(2023). Exclusive: ChatGPT traffic slips again for third month in a row. Reuters.
  17. Vaswani, A. et al.(2017). Attention Is All You Need. In Advances in Neural Information Processing Systems (NIPS), 30, 6000-6010.
  18. Wollowski, M.(2023). Using ChatGPT to produce code for a typical college-level assignment. AI Magazine, 44(1), 129-130. https://doi.org/10.1002/aaai.12086