• Journal of Practical Engineering Education
• /
• v.16 no.5_spc
• /
• pp.663-676
• /
• 2024

This study, from the perspective of Learning with AI, aimed to explore the educational applicability of writing evaluation criteria generated by artificial intelligence. Specifically, it sought to systematically analyze the similarities and differences between AI-generated criteria and those developed by humans. The research questions for this study were set as follows: 1) What characteristics do the writing evaluation criteria generated by AI tools have? 2) What similarities and differences exist between the writing evaluation criteria generated by humans and AI tools? GPT and Claude were selected as representative AI tools, and they were tasked with generating writing evaluation criteria for undergraduate students. These AI-generated criteria were then compared with human-created criteria. The results showed a commonality: Both humans and AI-tools placed the highest importance on categories related to content. However, while humans evaluated based on three main categories - content, organization, and language usage - the AI tools included additional categories such as format and citations, original thinking, and overall impression. In general, human tended to include more detailed items within each evaluation category, while AI tools presented more concise items. Notably, differences were observed in language-related aspects and scoring systems, which were influenced by the AI tools being developed based on English. This study offers important insights into the development of collaborative evaluation models between humans and AI, and it explores the potential role of AI as a complementary tool in educational assessment in the future.