• The Mathematical Education
• /
• v.63 no.2
• /
• pp.187-207
• /
• 2024

In this study, we explored the feasibility of automatically scoring descriptive assessment items using GPT-4 based ChatGPT by comparing and analyzing the scoring results between teachers and GPT-4 based ChatGPT. For this purpose, three descriptive items from the permutation and combination unit for first-year high school students were selected from the KICE (Korea Institute for Curriculum and Evaluation) website. Items 1 and 2 had only one problem-solving strategy, while Item 3 had more than two strategies. Two teachers, each with over eight years of educational experience, graded answers from 204 students and compared these with the results from GPT-4 based ChatGPT. Various techniques such as Few-Shot-CoT, SC, structured, and Iteratively prompts were utilized to construct prompts for scoring, which were then inputted into GPT-4 based ChatGPT for scoring. The scoring results for Items 1 and 2 showed a strong correlation between the teachers' and GPT-4's scoring. For Item 3, which involved multiple problem-solving strategies, the student answers were first classified according to their strategies using prompts inputted into GPT-4 based ChatGPT. Following this classification, scoring prompts tailored to each type were applied and inputted into GPT-4 based ChatGPT for scoring, and these results also showed a strong correlation with the teachers' scoring. Through this, the potential for GPT-4 models utilizing prompt engineering to assist in teachers' scoring was confirmed, and the limitations of this study and directions for future research were presented.

• Journal of The Korean Association of Information Education
• /
• v.25 no.6
• /
• pp.1005-1013
• /
• 2021

With the prolonged COVID-19, the existing academic gap is widening. The purpose of this study is to provide homeroom teachers with a visual confirmation of the academic achievement gap in grades and classrooms through academic achievement analysis, and to use this to help them design lessons and explore ways to improve the academic achievement gap. The data of students' Korean and math diagnostic evaluation scores at the beginning of the school year were visualized as clusters using the K-means algorithm, and as a result, it was confirmed that a meaningful clusters were formed. In addition, through the results of the teacher interview, it was confirmed that this system was meaningful in improving the academic achievement gap, such as checking the learning level and academic achievement of students, and designing classes such as individual supplementary instruction and level-specific learning. This means that this academic achievement data analysis system helps to improve the academic gap. This study provides practical help to homeroom teachers in exploring ways to improve the academic gap in grades and classes, and is expected to ultimately contribute to improving the academic gap.

• Communications of Mathematical Education
• /
• v.37 no.2
• /
• pp.233-256
• /
• 2023

This study conducted foundational research to derive ways to use ChatGPT in mathematics education by analyzing ChatGPT's responses to questions from the National Assessment of Educational Achievement (NAEA) and the College Scholastic Ability Test (CSAT). ChatGPT, a generative artificial intelligence model, has gained attention in various fields, and there is a growing demand for its use in education as the number of users rapidly increases. To the best of our knowledge, there are very few reported cases of educational studies utilizing ChatGPT. In this study, we analyzed ChatGPT 3.5 responses to questions from the three-year National Assessment of Educational Achievement and the College Scholastic Ability Test, categorizing them based on the percentage of correct answers, the accuracy of the solution process, and types of errors. The correct answer rates for ChatGPT in the National Assessment of Educational Achievement and the College Scholastic Ability Test questions were 37.1% and 15.97%, respectively. The accuracy of ChatGPT's solution process was calculated as 3.44 for the National Assessment of Educational Achievement and 2.49 for the College Scholastic Ability Test. Errors in solving math problems with ChatGPT were classified into procedural and functional errors. Procedural errors referred to mistakes in connecting expressions to the next step or in calculations, while functional errors were related to how ChatGPT recognized, judged, and outputted text. This analysis suggests that relying solely on the percentage of correct answers should not be the criterion for assessing ChatGPT's mathematical performance, but rather a combination of the accuracy of the solution process and types of errors should be considered.

• Journal of Platform Technology
• /
• v.11 no.5
• /
• pp.72-83
• /
• 2023

ChatGPT, commercial launch in late 2022, has shown successful results in various professional exams, including US Bar Exam and the United States Medical Licensing Exam (USMLE), demonstrating its ability to pass qualifying exams in professional domains. However, further experimentation and analysis are required to assess ChatGPT's scholastic capability, such as logical inference and problem-solving skills. This study evaluated ChatGPT's scholastic performance utilizing the Korean College Scholastic Ability Test (KCSAT) subjects, including Korean, English, and Mathematics. The experimental results revealed that ChatGPT achieved a relatively high accuracy rate of 69% in the English exam but relatively lower rates of 34% and 19% in the Korean Language and Mathematics domains, respectively. Through analyzing the results of the Korean language exam, English exams, and TOPIK II, we evaluated ChatGPT's strengths and weaknesses in comprehension and logical inference abilities. Although ChatGPT, as a generative language model, can understand and respond to general Korean, English, and Mathematics problems, it is considered weak in tasks involving higher-level logical inference and complex mathematical problem-solving. This study might provide simple yet accurate and effective evaluation criteria for generative artificial intelligence performance assessment through the analysis of KCSAT scores.

• The Mathematical Education
• /
• v.62 no.3
• /
• pp.381-400
• /
• 2023

As the era of solving various and complex problems in the real world using artificial intelligence and big data appears, problem-solving competencies that can solve realistic problems through a mathematical approach are required. In fact, the 2015 revised mathematics curriculum and the 2022 revised mathematics curriculum emphasize mathematical modeling as an activity and competency to solve real-world problems. However, the real-world problems presented in domestic and international textbooks have a high proportion of artificial problems that rarely occur in real-world. Accordingly, domestic and international countries are paying attention to the reality of mathematical modeling tasks and suggesting the need for authentic tasks that reflect students' daily lives. However, not only did previous studies focus on theoretical proposals for reality, but studies analyzing teachers' perceptions of reality and their competency to reflect reality in the task are insufficient. Accordingly, this study aims to analyze in-service mathematics teachers' perception of reality among the characteristics of tasks for mathematical modeling and the in-service mathematics teachers' competency for designing the mathematical modeling tasks. First of all, five criteria for satisfying the reality were established by analyzing literatures. Afterward, teacher training was conducted under the theme of mathematical modeling. Pre- and post-surveys for 41 in-service mathematics teachers who participated in the teacher training was conducted to confirm changes in perception of reality. The pre- and post- surveys provided a task that did not reflect reality, and in-service mathematics teachers determined whether the task given in surveys reflected reality and selected one reason for the judgment among five criteria for reality. Afterwards, frequency analysis was conducted by coding the results of the survey answered by in-service mathematics teachers in the pre- and post- survey, and frequencies were compared to confirm in-service mathematics teachers' perception changes on reality. In addition, the mathematical modeling tasks designed by in-service teachers were evaluated with the criteria for reality to confirm the teachers' competency for designing mathematical modeling tasks reflecting the reality. As a result, it was shown that in-service mathematics teachers changed from insufficient perception that only considers fragmentary criterion for reality to perceptions that consider all the five criteria of reality. In particular, as a result of analyzing the basis for judgment among in-service mathematics teachers whose judgment on reality was reversed in the pre- and post-survey, changes in the perception of in-service mathematics teachers was confirmed, who did not consider certain criteria as a criterion for reality in the pre-survey, but considered them as a criterion for reality in the post-survey. In addition, as a result of evaluating the tasks designed by in-service mathematics teachers for mathematical modeling, in-service mathematics teachers showed the competency to reflect reality in their tasks. However, among the five criteria for reality, the criterion for "situations that can occur in students' daily lives," "need to solve the task," and "require conclusions in a real-world situation" were relatively less reflected. In addition, it was found that the proportion of teachers with low task development competencies was higher in the teacher group who could not make the right judgment than in the teacher group who could make the right judgment on the reality of the task. Based on the results of these studies, this study provides implications for teacher education to enable mathematics teachers to apply mathematical modeling lesson in their classes.