Teacher's Perception for Korean's Achievement Standards-Based Testing System and Evaluation Method of Learners' Academic Ability

Abstract

The purpose of this research is to evaluate teachers' perception of Korean's achievement standard-based testing system (ASTS) and its process of implementation, and to propose a method of evaluating students' academic aptitude based on university entrance examinations. The core of the 2015 Revised National Curriculum is asserted by changes in classroom instruction, specifically encouraging students' participation in class based on a new method to evaluate student achievement. A total of 124 teachers in charge of student career counseling in middle and high schools in the Jeonbuk province participated in the study. The schools implementing the new method of ASTS were using 61.6% for unit school cut-off point, as opposed to the existing fixed cut-off point of 38.4%. The teachers understanding of the achievement evaluation method was rated 3.54 on the 5-point Likert scale, implying that they had a relatively good understanding of the method. Some of the challenges associated with reflecting the scores from the new student ASTS include difficulty of comparing scores across schools; grade inflation; advantages and disadvantages associated with the type of high school; and the increased importance of university entrance examination. In the ASTS, the fairness during the evaluation of the high school grades and the consequently the reliability of the evaluation prove worrying. As an ultimate result, selecting students based on university admissions data became untrustworthy. There should be further discussions on how students' achievement obtained from the new ASTS should be applied during the university admission process and how students' academic aptitude can be assessed in order to set a direction for secondary school education.

1. INTRODUCTION 

Student academic achievement system for secondary education in Korea has undergone constant evolution since the 1980s, experimenting on both relative and absolute achievement systems. The preceding system evaluated students for university entrance admission based on their achievement in the national university entrance examination, individual university examinations, or on the academic aptitude test. The relative assessment system remained main stream, as the importance of high school academic achievement increased so as to allow the Korean universities to play a greater role in student admission [1]. However, despite the advantages of relative assessment, the assessment method mostly relied on pen-and-paper tests too much, and student competition was overheated. For these reasons, the relative assessment system has been criticized; another concern was that it prevented students from developing cooperation, creativity, and character [2], [3]. In addition, in the relative assessment system, it was difficult to obtain a precise level of a students’ academic achievement due to the lack of information on the extent to which students understood subject materials; other additional information on students’ academic aptitude was lacking too. Therefore, the relative assessment system has several fundamental limitations in terms of evaluating students’ potentials and talents. In this context, in the recent years, based on the Enhancement Plan for Secondary School Education [1] and the 2015 Revised National Curriculum [3], the Ministry of Education has been working on establishing a new direction for secondary education and innovating the student achievement assessment method that would promote creativity and character education, as well as maximizing students’ potential. 

Accordingly, the ministry created a new assessment system that avoids the competition-driven education of the past and provides education environment that allows students to develop their creativity and character. It was introduced to middle and high school textbooks in 2012 in phased introduction and, in 2014, to the normal curriculum of high schools. Today, this new system is generally applied to all secondary education curricula. Beginning with the Korean history class, the ASTS was applied to university academic aptitude test in 2017 and, in 2018, expanded to include the English class. Due to the introduction of new testing system, fundamental changes are now underway in high school class and testing system in general. 

As Korea goes beyond the knowledge-based society to face the 4th Industrial Revolution era, there is an increasing demand for enhancing national competitiveness and creativity development through school education [4]. Future society requires that young professionals are able to understand and effectively solve social, cultural, and technological problems that are more complex than those of today. Students need the ability to integrate knowledge and function and apply them flexibly under various contexts and situations. Accordingly, the 2015 Revised National Curriculum was designed to allow students to cultivate basic aptitudes in humanities, society, and science & technology so that they could become ‘creative convergence type talent’ equipped with imagination and creativity both in humanities and natural/applied sciences. In this context, the present study aims to investigate the current status of the secondary school student ASTS and the level of teachers’ understanding of this system; our second goal is to explore the possibility to differentiate student academic aptitude in preparation for future society. 

 

2. THEORETICAL BACKGROUND 

The ASTS is introduced to overcome the limitations of the existing relative evaluation, to provide specific information on the students' academic achievement level, and to improve their learning ability by providing various learning opportunities appropriate to the achievement level [4]. In addition, the ASTS is an absolute evaluation system that rules out excessive competition among students that a necessitated by relative evaluation. Therefore, the ASTS is a transition from a relative sequence-based normative reference evaluation (relative evaluation) to a goal-based reference evaluation (absolute evaluation) [4]. Achievement standard includes the knowledge, function, and attitude to be achieved by students in each curriculum and determines the scope and level of these parameters. Thus, it is a practical guideline on what teachers should teach and what students are expected to learn, so it is used as a basis for classroom instruction. Achievement level is determined by categorizing students’ academic achievement for each class into 3 levels (A-B-C) or 5 levels (A-B-C-D-E). The ASTS differs from the relative system currently used at school (9-level ranking system) in terms of the purpose, scoring method, and expected assessment activities. The absolute academic achievement system for high school emphasizes a “curriculum that is creative, student-oriented, diverse and customized” [5], thereby reducing overheated competition among students that is inherent in the existing relative assessment system. However, despite the positive aspects of this new ASTS, negative perception of this new system persists at schools today, due to sensitivity to school academic achievement records, grade inflation that appeared in the absolute assessment system of the past, distrust in educational policy caused by frequent policy changes, and so on [1]. In these circumstances, it is necessary that the high school ASTS adopts the system that allows students to experience the process of exploring various subjects through proactive participation; furthermore, the learning and growth resulting from such process should be verified [6, 7, 8]. However, if process-centered absolute assessment system is to be used with the emphasis on helping students build creativity and character, university entrance system that selects students based on the results of assessment only will face difficulty quantitatively evaluating student’s academic ability. 

 

3. RESEARCH METHODS 

In the present study, in order to derive a plan for academic aptitude assessment in university entrance examination, both a literature review and a questionnaire survey were performed. Through the literature review, we sought to analyze the current status of the student ASTS and consider relevant case studies; at this stage, a preliminary research to review how the system was studied and used was also undertaken. Furthermore, to assess the level of teachers’ overall perception of the system, a questionnaire survey was conducted on a total of 124 in-service counseling teachers in middle/high schools in the Jeonbuk province(150 teachers were using the Internet survey tool 'Survey Monkey' and selected a case that responded faithfully). The survey results were analyzed with the focus on perception and implementation of the system, as well as its use in the university entrance process. 

 

Table 1. Questions on teacher survey on the academic ASTS

 

4. RESULTS and DISCUSSION 

4.1. Teacher survey on the ASTS 

A questionnaire survey was created to collect data needed to assess students’ academic aptitude based on high school academic achievement assessment records. The questionnaire consisted of 23 questions: 6 questions on the level of understanding of the academic ASTS; 7 questions on its current status; and 10 questions on suggestions for how to use the system in the university entrance process (see Table 1 for the major questions). The survey was conducted online from September 29, 2017 to October 15, 2017. 

 

4.2 Level of teachers’ understanding of the academic ASTS and current status 

The teachers had a good understanding of the system, with the mean score amounting to 3.54(cronbach's α=.82) on a 5-point Likert scale scoring (Table 2). Furthermore, the teachers scored 3.60 on understanding the purpose and direction of the system, and 58.9% (73) of them understood these two aspects well or very well. In addition, the teachers scored 3.55 on average in terms of understanding the achievement standard for each class subject, with 53.2% (66) of them understanding them well or very well. Furthermore, the teachers scored 3.46 on understanding of the expected effect of implementing the system, with 48.4% (60) understanding them well or very well. With regard to the perception of the system, the average score were 3.5-3.6, with 48-60% of the teachers understanding them well; by contrast, 11~15% of them less or not at all. 

 

Table 2. Teachers’ understanding of academic ASTS

 

Taken together, the results suggest that teachers, who will play the leading role in a successful implementation of the system, need training and education to obtain the overall understanding of the system. Although the teachers’ understanding of achievement standard and achievement level was good (3.20~3.35), the level of student feedback on the achievement results was poor (2.63). Due to extended work hours and a high number of students they are in charge of, the teachers do not receive sufficient feedback from their students. The results also demonstrate that evaluation of the learning process, as well as the assessment of the process to achieve the standard and individual guidance, are not being done properly. The successful implementation of ASTS is consistent with the study by Lim and Jeoung (2010) that teachers should understand and actively participate through training and education [8]. In the classroom and assessment process, individual guidance for students and formative evaluation in accordance with achievement standards should be facilitated in order to reduce learning burden of teachers [1]. 

 

4.3 Effects of implementing the academic ASTS

Multiple responses on the effect of the ASTS as compared to the existing system (relative assessment) were analyzed. The teachers scored 3.55(cronbach's α=.91) in terms of their understanding of the positive effect of the assessment system (Table 3). In addition, they recognized the following other positive effects: increased attention to individual student reaching academic achievement (42.7%); enhanced classroom instruction due to the clearer achievement objective (37.1%); less stress on students due to mitigated student competition (37.1%); customized classes based on the student level (36.3%). There was an opinion that, in contrast to the relative assessment system where teachers did not pay attention to the average score in classes, under the new system, they paid more attention to class averages and frequently provided feedback to individual student achievements. There was another opinion that, in order for the new system to be properly established at school, it is necessary to improve the quality of teachers’ assessment skills, innovate the university entrance admission system, and introduce changes to classroom instruction. From the students' perspective, the positive effect brought about less stress in study (46%), enhanced self-motivation due to an increased sense of achievement (38.7%), enhanced self-led learning ability (32.3%), active classroom participation (26.6%), and enhanced creativity (15.3%). 

 

Table 3. Positive effects from implementing the academic ASTS

 

achievement standard and level (42.7%); grade inflation (44.4%); difficulties in developing assessment tools (performance assessment, written assessment) (39.5%); decreased quality in classroom instruction and assessment due to increased teacher load (31.5%); difficulty in securing objective academic achievement records needed for university entrance (40.3%); expecting university to avoid using school achievement record as admission criteria (32.3%); and ambiguity of determining achievement standard (27.9%). Other opinions included problems associated with determining the level of difficulty by school and concerns about grade inflation. 

 

4.4 Status of the ASTS 

Our results suggest the following difficulties experienced by teachers under the ASTS (Table 4): developing and applying achievement assessment standard and level for different class appropriate for schools (54.8%); student feedback based on achievement results and reinforcement classes (38.7%); process of developing assessment questions according to achievement criteria and level (36.3%); planning and implementing based on achievement standard and criteria (29.8%); and planning and implementing performance assessment based on achievement standard and level (28.2%). Other opinions included difficulties in learning that result from the differences in class level; difficulties in assessment due to the large number of students; and difficulties in the operating assessment system apart from university entrance preparation. 

 

Table 4. Difficulties associated with operating the ASTS

 

The positive changes observed at school after the implementation of the ASTS are as follows (Table 5): student-oriented teaching, more meaningful classroom activities (30.7%); performing the projects that allow students to cooperate, collaborate, and discuss (21.0%); use of various classroom instruction methods (17.0%); students’ active participation in class (13.7%); and balanced classroom activities (4.0%). However, the negative effects mentioned by our respondents are that the assessment questions were becoming easier and irrelevant to university entrance preparation and that students’ motivation declined after achieving the achievement standard. 

 

4.5 Using the ASTS in university entrance application 

The schools implementing the ASTS were using 61.6% for unit school cut-off point, as compared to the fixed cut-off point of 38.4%. The use of unit school cut-off point could lead to grade inflation that, in turn, may lead to distrust in high school academic scores for university admission. The survey results show that the teachers responded with “somewhat” to the question about their concern about grade inflation, while 10.5% responded “serious” or “very serious”. If universities admit students based on academic achievement assessment in high school, it does not accurately reflect the true academic aptitude of students, and, therefore, an alternative solution is needed for the university admission process. 

 

Table 5. Changes caused by the implementation of the ASTS

 

The problems associated with reflecting the results of the ASTS in the university admission process include the following: difficulties of comparing cut-off points across schools (40.3%); decline in reliability due to grade inflation (32.3%); advantages and disadvantages due to high school type (20.2%); reinforcing university preparation tests. The highest percentage of respondents answered that comparing schools were most difficult, which seems to reflect the concern for increasing distrust in high school academic scores(Table 6). On the other hand, they selected the standardized score (45.2%), class subject achievement level (24.2%), original score (8.9%), transformed score (16.9%), and others (4.8%) as most appropriate parameters in terms of using assessment score as university admission criteria(Table 7). In order to overcome the  weaknesses of ASTS, an alternative method to measure the academic ability of the examinee is required in the university entrance examination [1], [3].

 

Table 6. The most worrying comments in the process of reflecting ASTS scores at university entrance examination

 

Table 7. Evaluation factors of university entrance using ASTS scores

 

5. CONCLUSION 

The results of our survey on the perceptions of middle/high school university entrance counseling teachers of using the ASTS demonstrate that the teachers understood the system fairly well (the average score 3.54 on a 5-point Likert scale). However, since teachers are responsible for successfully operating the assessment system, they should have a deeper understanding of the purpose and direction of the system and the achievement standard. The problems associated with applying the scores from the ASTS to university entrance process include the difficulties of cross-school comparison, grade inflation, advantages and disadvantages according to high school type, and reinforced university exams.

Compared to the existing relative assessment system where students are ranked by subject based on the 9-grade system, the ASTS has the advantage of reducing stress on students and increasing quality of education. However, the new system lacks the ability to differentiate when applied to university admission process, which could lead to a decline in reliability of school academic records and fairness. Since high school curriculum largely depends on the university admission process, the ASTS is meaningless if the current 9-grade relative assessment system remains in effect. However, either the fixed score system or the cut-off score system is used to assign achievement assessment grade. In the latter case, the differences in original scores vary widely across schools, compromising objectivity and, as a result, leading to the problem of incorporating cross-school differences. Therefore, universities should seriously consider how to apply school academic scores obtained under the ASTS and find more effective ways to evaluate students’ academic aptitude. Furthermore, when applying school academic records to the university entrance process under the ASTS, it should be determined whether or not the total score of all classes or individual subject grades should be used. If the total scores are used for school academic records, students will try to do their best in all subjects, rather than focus on certain classes. However, in this case, it is difficult to identify in which subject(s) students excel and what their specialties and talents are. To solve this problem, we can calculate the scores for each subject, which has the advantage of allowing universities to select students who would outperform in a specific major. However, school subjects that are not considered during the university admission process are likely to be strategically ignored by students, which is not in line with the educational philosophy of the 2015 Revised National Curriculum which aims at convergence education and character education.

Under the current university entrance system in Korea, which is so fiercely competitive, the system is very likely to emphasize its selective function than educational function. Since performance in school academic records is used as the core criterion in the university admission process, students are likely to focus more on school activities than on school grades, while other educational activities that are not reflected in the grades will be ignored. As a result, such program is likely to yield poor performance. Therefore, there remains a concern that replacing the current 9-grade relative student assessment system in high school with the absolute assessment 3-5 grade system, while leaving the student assessment standard during university entrance process relative and the hierarchical structure of university unchanged, will not produce any significant effect. Although the absolute assessment system is ideal for student assessment from the philosophical point of view, in the context of fierce competition in university entrance and the hierarchical university ranking system, it could result in grade inflation, decline in academic aptitude, and persistent academic performance-focused study.

The reason why school academic records were calculated using the absolute assessment method is because it mitigates excessive competition among students and create more cooperative learning structure. However, the absolute assessment system weakened the trust in fairness and objectivity of the system via grade inflation. In order to overcome this problem, it is necessary to enhance the high school education system, including changes in the university entrance system suitable for the ASTS and high school achievement scoring system, as well as renewed university admission system that supports them. In particular, considering the current university entrance preparation system entrenched in the relative assessment system (that picks winners), a more reasonable assessment system that can supplement the lack of discriminating power in the ASTS while broadening the range of performance criteria in the university admission process system (university entrance test score) in achievement assessment should be introduced.