• Journal of the Korean Society of Earth Science Education
• /
• v.17 no.2
• /
• pp.181-193
• /
• 2024

Summative assessment provides information on how well students have achieved learning objectives, making the development of high-quality assessment items essential for accurate evaluation. This is one of the competencies that teachers must possess. This study aims to analyze summative assessment items created by pre-service elementary teachers, examining their intentions and the difficulties encountered in the item development process. The study involved 45 second-year students enrolled in an elementary teacher training university. They were grouped into teams of three and tasked with developing ten items, documenting the purpose of each item, the answer key, and the challenges faced during item creation. The collected summative assessment items were analyzed using a two-dimensional purpose classification table that includes Klopfer's taxonomy of educational objectives. The intentions behind the summative assessments and the difficulties faced during item development were inductively organized and analyzed through qualitative data analysis. The results revealed that pre-service elementary teachers adequately reflected scientific content elements but did not evenly cover assessment domains. The most challenging aspect for them was adjusting the difficulty level. Although they considered most factors that should be taken into account during item development, these considerations were not reflected in the actual items. These findings suggest that knowledge and experience are crucial in developing summative assessment items, and systematic lectures are necessary for pre-service elementary teachers.

• Journal of The Korean Association For Science Education
• /
• v.22 no.2
• /
• pp.345-356
• /
• 2002

This study investigated the trends in the number of applicants and mean score and applicants' responses on the test items of 'science inquiry' of the College Scholastic Ability Test(CSAT) implemented for 3 years$(1999\;{\sim}2001)$. The results of this study were as follows: The percentage of applicants of science track for 1995 CSAT were 43.13%, but reduced to 29.5% for 2001 CSAT. And unlike other tracks, the percentage of male applicants, ranking above average, of science track was 65.58%, which is about twofold of female applicants(34.42%). The mean score of 'Science inquiry' was 58.6 in 1999, and 69.5 in 2001. And the score of the applicants, ranking above average, of humanity and social science course and science course, were 85.8 and 90.7 respectfully in 2001 CSAT. These high mean scores were caused by the policy of "easy CSAT" so called. Most of test items were developed to have difficulty 60-79% or above 80%. This easy CSAT provoked intense dispute about the discriminating power of CSAT. The mean score of male applicants was higher than that of female. But the difference decreases every year. Applicants were generally very good at solving tests focusing on process skills only but poor at solving tests related to physics or calling for two or more science concepts. Thus special measures to cope with the decrease in applicants, especially female applicants, for science track should be provided. To increase discriminating power of CSAT, it is recommended to develop test items with wider range of difficulty and to reduce test items which are focussing process skills and can be solved without any special science concepts. And special consideration should be given to teaching the content area with poor achievement and high actual difficulty compared to the expected.

• Journal of the Korean Chemical Society
• /
• v.47 no.5
• /
• pp.505-518
• /
• 2003

A survey has been administered to investigate the adequacy of the chemistry teacher employment examination. The questionnaires were sent to the professors of the fourteen departments of chemistry education all over the nation and the responses were collected from the thirteen departments. From the analyses of the responses, the problems were pointed out and the suggestions were made for the improvement of the examination to recruit excellent chemistry teachers. Reconsideration of the ratio of subject matter education to subject contents, adjustment of format and number of questions, maintaining of proper degree of difficulty and discrimination of items, reconsideration of relevancy to practical education, raising the number of committee for the examination, the reflection of the 7th science curriculum, and opening to the public of the information related to the teacher employment examination were discussed.

• Journal of The Korean Association For Science Education
• /
• v.35 no.1
• /
• pp.121-130
• /
• 2015

Identifying students' achievement level and reflecting it on educational policy making or instructional improvement by analyzing the results of nationwide standardized assessment is an integral part of accountability in education. On the basis of this premise, we analyzed the characteristics of National Assessment of Educational Achievement (NAEA) items for middle school science subject conducted from 2010 to 2013 by using the option response rate distribution curve, the fittest graph estimated from the response rate of correct/incorrect options by achievement score. Furthermore, we classified the type of option response rate curve in terms of correct and incorrect options. Results of the analysis of option response curve showed that five types of correct option response curve (S-shaped, J-shaped, straight-shaped, F-shaped, and step-shaped) and 4 types of incorrect option response curve (down-slope, flat, mound, and up-slope) were identified. The most common type of items was the combination of S-shaped correct option response curve and down-slope incorrect option response curve, which are considered as appropriate items to discriminate the students according to achievement level. Moreover, correct option response was found to be correlated with incorrect option response. Based on the results, we also discussed some implications on teaching-learning method and classroom assessment in science education.

• Journal of the Korean earth science society
• /
• v.37 no.3
• /
• pp.173-185
• /
• 2016

The purpose of this study is to investigate the effects of systems thinking on high school students' self-regulatory efficacy and self-confidence that constitute science self-efficacy. We set self-regulatory efficacy as a factor of students' systems thinking affects their self-confidence on science through self-regulatory efficacy. A total of 210 students were sampled from general high schools and 188 valid cases were analyzed. The instrument has 39 items that consist of 20 items measuring systems thinking and 19 items of science self-efficacy. The result of the exploratory factor analysis indicated that 20 items for systems thinking, 8 items for self-regulatory efficacy, 4 items for self-confidence are reasonable. Testing the goodness of fit of a structural equation model, it turn out to be appropriate (${\chi}^2$=344.498, df=242, TLI= .921, RMSEA= .044) using 24 items (mental model, personal mastery, systems analysis, self-regulatory efficacy, and self-confidence were constructed). In addition, the mental model, which is one factor of systems thinking, is mediated by self-regulatory efficacy that affects self-confidence directly and/or indirectly. The results suggest that systems thinking affects science self-efficacy directly and indirectly. Utilizing systems thinking in science education can produce a theoretical basis in improving students' confidence and self-efficacy about science.

• Journal of The Korean Association For Science Education
• /
• v.24 no.3
• /
• pp.519-531
• /
• 2004

In recent years, there has been an increased emphasis on performance assessment to evaluate students' abilities. Our nation has introduced a change in testing and assessment. Additional work on the efficacy, reliability, and comparability in order to develop the performance assessment item has been needed in the enforcement of the 7th National Science Curriculum. Also, criteria for professional and technical standards has been needed to be developed. The purpose of this study was to draw out various key concepts and to develop achievement standards, assessment standards and performance assessment items based on the 7th National Science Curriculum on the subject matter of reproduction(chapter 13) and biological accumulation(chapter 17). And also, this study examined the validity of completed performance assessment items based on classical test theory and polytomous item response theory. Twelve key concepts in chapter 13(reproduction) and four from chapter 17(biological accumulation) were abstracted. Twenty-six achievement standards in chapter 13(reproduction), and nine in chapter 17(biological accumulation) were developed. The achievement standards were determined in terms of knowledge(K), process skill(P) and attitude(A). Twenty-five assessment standards in chapter 13(reproduction) and nine in chapter 17(biological accumulation) were developed. Based on the developed achievement standards and assessment standards, twenty-two performance assessment items(seventeen open-ended questions, three essays, and two portfolios) with concrete grading criteria were developed. Eight open-ended items were applied to 240 10th graders to evaluate reliabilities of the test which consisted of four items per each chapter. The results would be suggested that the applied items were valid for performance assessment because item difficulties and item discriminations were proper. There was not much differences in item discrimination between interpretation from classical test theory and that from polytomous item response theory. However, there were some differences in item difficulties between the interpretations of two theories because the characteristics of examinees were reflected in classical test theory.

• Journal of Science Education
• /
• v.44 no.1
• /
• pp.38-49
• /
• 2020

The purpose of this study is to analyze the paper test items of high school physics I 'Space-Time and the Universe' unit and how similar the textbook goals are to the cognitive and content areas. This study analyzed paper test items used from nine high schools and KICE about the unit. In order to analyze the distribution of items, they were classified by contents and by cognitive level which is knowledge, comprehension and application levels. As conclusion, application level items were a few, and unbalanced contents distribution appeared in school items. In comparison through the ratio difference index, the objectives of the textbook and the school items were very similar, but the similarities were low when comparing the school items with the KICE items. Based on the above, we suggested some ways to increase the validity of test items.

• Journal of Gifted/Talented Education
• /
• v.22 no.1
• /
• pp.117-139
• /
• 2012

This study compared the cognitive conflict on peer instruction concerning the level of difficulty in question, between middle school science gifted students and non-gifted students. For the study, 35 the 7th grade science-gifted students in the organization affiliated with Science Education Institute for the Gifted in Seoul and Gyeong-gi province, and 14 the 8th grade science-gifted students, and 71 the 8th grade students. They performed peer instruction on propagation of straight light and composition of light and then, discussed three concept problems. After discussing the students took paper pencil test about changing levels of cognitive conflict. Regardless of the level of difficulty in question, the science-gifted students showed meaningful decreased figures on cognitive re-evaluation factors after peer-discussion. They trusted their peers, so during discussion, they explained their concepts. Furthermore discussion process enabled them to do reflective thinking. consequently, discord of students dropped, and total figures of cognitive conflict also declined. Science-gifted students have a tendency to worry lower than general students, though they felt anxiety as difficulty of the problems after peer-discussion. Through peer-discussion, science-gifted students presented statically decreased anxiety factors. By means of analyzed results of changing cognitive conflict of science-gifted students, developing and adapting strategies of cognitive conflict considering learner characteristics of science-gifted students is needed.

• Journal of The Korean Association For Science Education
• /
• v.26 no.4
• /
• pp.492-501
• /
• 2006

Gender differences in TIMSS 2003 science achievement by item type, benchmark, and content area were examined by producing a Gender Differences Index (GDI) in this study. International trends identified that male students performed better than female students in TIMSS 2003 science achievement in all types of items. The overall achievement of Korean male students was better than Korean female students, especially in multiple-choice type items. Male students outperformed females in three benchmarks, including advanced, high, and intermediate international benchmark, but they did not outperform females in the low international benchmark when gender differences of the international average as well as the Korean average were taken into account. The results of the analysis of the international average and the Korean average by content area showed that gender differences were the greatest in earth science and smallest in chemistry. In life science, female students excelled when considering the international average while male students excelled when considering the average of Korean students' performance. In addition, the number of items in which male students outperformed females was larger in both factual knowledge and the conceptual understanding domain. Implications for reducing gender differences in science achievement in Korea based on the results were provided.

• Proceedings of the Korean Society for the Gifted Conference
• /
• 2005.05a
• /
• pp.197-216
• /
• 2005

정보과학영재의 선발에서 가장 중시되어야 할 문제는 교육에 의한 가능성을 측정하는 것이다. 그러므로 정보과학영재의 선발에서는 정보과학영재의 정의와 특성에 따라 그들이 가지고 있는 정보과학분야에 대한 창의적 사고력과 이해력이 바탕이 되어야 한다. 본 논문에서는 영재와 정보과학영재에 대한 여러 학자들의 정의와 특성을 바탕으로 초등정보과학영재 선발평가에 필요한 필수요소를 추출하여 선발평가 문항을 개발하였다.