• Korean journal of medical education
• /
• v.30 no.4
• /
• pp.347-357
• /
• 2018

Purpose: The purpose of this study was to apply alternative standard setting methods for the Korean Medical Licensing Examination (KMLE), a criterion-referenced written examination, and to compare them to the conventional cut score used on the KMLE. Methods: The process and results of criterion-referenced standard settings (i.e., the modified-Angoff and bookmark methods) were evaluated. The ratio of passing and failing examinees determined using these alternative standard setting methods was compared to the results of the conventional criteria. Additionally, the external, internal and procedural evaluation of these methods were reviewed. Results: The modified-Angoff method yielded the highest cut score, followed sequentially by the conventional method and the bookmark method. The classification agreement between the modified-Angoff and bookmark methods was 0.720 measured by Cohen's ${\kappa}$ coefficient. The intra-panelist classification consistency of modified-Angoff method was higher than bookmark method. However, the inter-panelist classification consistency was vice versa. The standard setting panelists' survey results showed that the procedures of both methods were satisfactory, but panelists had more confidence in the results of the modified-Angoff method. Conclusion: The modified-Angoff method showed results that were more similar to those of the conventional method. Both new methods showed very high concordance with the conventional method, as well as with each other. The modified-Angoff method was considered feasible for adoption on the KMLE. The standard setting panelists responded positively to the modified-Angoff method in terms of its practical applicability, despite certain advantages of the bookmark method.

• Communications of Mathematical Education
• /
• v.22 no.2
• /
• pp.109-124
• /
• 2008

A characteristic of the national mathematics curriculum revised in 2007 is to repeal the level-oriented individualized curriculum and choose substance of individualized teaching and learning based on the student's achievement level and quality. To do this we first have to think through how to compare students' achievement and differentiate classes. In this paper, we introduce the (modified) Angoff method as a method for comparing students' achievement and the concept of representative items for each achievement level in the National Assessment of Educational Achievement of Mathematics, and discuss how to use them in individualized teaching and learning, especially comparing students' achievement.

• Journal for History of Mathematics
• /
• v.24 no.3
• /
• pp.145-163
• /
• 2011

This study sets the cut points of affective achievement scores based on the criteria referenced assessment. The modified Angoff method is applied to the standardized mathematics affect inventory which had validity and reliability. The cut points are set for 6 factors i.e. learning directivity, self control, anxiety, interest, cognizing value and confidence. As the results, among percentages of factor that middle school 2nd grade students in Korea achieved, the proportion of cognizing value is the highest. And there are no difference of the proportions as for gender, differentiated instruction, and region.

• Education of Primary School Mathematics
• /
• v.16 no.2
• /
• pp.163-182
• /
• 2013

The purpose of this study is an analysis on the 3rd graders' achievement in the elementary school. For this purpose, this study, first, analysed on the 3rd graders' achievement like the ratios of the achievement levels for whole students, sexual students, and regional students. Second, this study analysed the 3rd graders' assessment results like the total averages, averages for the contents area, sexual students, and regional students. Third, this study analysed students' special responses on the items.

• School Mathematics
• /
• v.6 no.3
• /
• pp.301-312
• /
• 2004

The aim of the National Assessment of Educational Achievement (NAEA) is to produce specific and reliable resources required for the diagnosis and quality control of teaching and learning by measuring the level of students achievement based on the national curriculum. In 2002, we introduced ‘modified Angoff Method’ to obtain more systematic and rational results about the achievement levels. The result indicated the differences of achievement level according to the differences of sexes. Female students achieved higher scores than male students in Grade 6. Male students achieved higher scores than female students in Grade 9 and 10. Furthermore it disclosed a problematic phenomenon that students in small towns and rural areas showed significantly lower scores in all six sub-areas of Mathematics compared with students in metropolitan and cities. The results from the NAEA listed above could be used as the authentic data for improving national curriculum and teaching and learning methods, the establishment of educational policies, and many other areas.

• School Mathematics
• /
• v.7 no.1
• /
• pp.33-54
• /
• 2005

One of the major results of National Assessment of Educational Achievement in 2003 is that the final test results were reported by four levels such as Advanced, Proficient, Basic, and Below Basic. We used it to investigate into the characteristics of students according to achievement levels, and finally we listed them. We think that these results will be a help to instructions according to achievement levels.

• Journal of Engineering Education Research
• /
• v.26 no.6
• /
• pp.40-50
• /
• 2023

This study is to examine the validity of assessing basic-level proficiency in physics among incoming engineering freshmen through item analysis and standard setting. For empirical analysis, we examined the physics subject taken by the freshman class of 2021 at K University, considering its significance for engineering students. In this study, we initially performed item analysis utilizing both classical test theory and item response theory. Subsequently, leveraging the item and test information, we employed a modified Angoff method and the Bookmark method for standard setting. Consequently, the difficulty level initially set during item development was found to be higher than the actual performance level exhibited by the students. This study highlights a discernible disparity between the expected university standard and the real proficiency level of incoming freshmen in terms of basic academic ability in physics. Based on these research findings, a comprehensive discussion on the fundamental academic competence of engineering students was conducted, underscoring the necessity for formulating a tailored learning approach leveraging the outcomes from the basic ability test.