• Journal of The Korean Association For Science Education
• /
• v.23 no.3
• /
• pp.265-275
• /
• 2003

The purpose of this study is to investigate the differences of science achievement between female- and male students in different assessment types, expecting the reconsideration of their scientific ability. to accomplish this purpose, the school science achievement score of 1,487 female- and male students in 2000 were analyzed from the perspectives of assessment methods and gender differences, achievement level and gender differences, and gender characteristics in correlation between each assessment type and total science score. The science achievement score in this study includes mi-term and final paper-and-pencil test in two semesters and performance assessment. The results are as follows. First, in the total science achievement score, whereas female students achieved significantly better than male counterparts in the 7th grade, there was no significant difference between two genders in the 8th and 9th grades. Second, as for the close-ended items, all students, regardless of their gender and grade, showed no significant difference. As for the open-ended items, whereas female students achieved significantly better than male counterparts in the 7th grade, there was no significant difference between gender in the 8th and 9th grades. As for the performance assessment, all female students, regardless of their achievement level, achieved significant better than male counterparts, which is very noteworthy. Third, in the uppermost achievement level, there was no significant difference not only in close- and open ended items but also in performance assessment. Fourth, male students' correlation between performance assessment and total science score was relatively higher than female students'. The result of this study, that female students did significantly better than male students in performance assessment, is interesting in that performance assessment is widely accepted as more reliable and valid method in measuring students' various ability than traditional paper-and-pencil test.

• Journal of The Korean Association For Science Education
• /
• v.20 no.1
• /
• pp.77-87
• /
• 2000

The purpose of the study is to understand the change process of middle school students' physics conceptions by the presented types of conflict situations. 274 middle school students were selected from one school in Pusan, however 257 students were participated in all the procedure of the study. After we classified students' physics conceptions into scientific and unscientific conceptions, presented three types of conflict situations. In this study three different cognitive conflict strategies were adopted; the first one is logical arguments(LCS: logical conflict situation), the second is actual demonstration(DCS: demonstrational conflict situation), and the third is two strategies together(DLCS). In this study, first, we investigated the change process of students' physics conceptions by three types of conflict situations. Second, we compared the effect of three conflict situations presentation, which includes positive effect by conceptual change from misconception to scientific conception and negative effect by conceptual change from scientific conception to misconception. Third, we studied characteristics of conceptual change by characteristics of conflict situations. In result, DLCS group and DCS group were more positive effect than LCS group in mechanics, DLCS group and LCS group were more positive effect than DCS group in electricity. It seems that mechanics are closely related to physical experiences, while electricity are more abstract.

• 한국노년학
• /
• v.30 no.4
• /
• pp.1095-1106
• /
• 2010

The purpose of this study was to develop a Abdominal Obesity Elderly(AOE) Health Status Scale to assess self rated health level. The subjects of this study were total 255(pretest 56, test 199) elderly over 65year. The data were collected between June to August, 2009. The method of this study as follow: items generation, test of validity and reliability, 5 panels of experts reviewed the preliminary questionnaire and then data were collected from elderly. Factor analysis, Cronbach's alpha, mean, standard deviation were analyzed the data by SPSS-Win 12.0 program. Results of factor analysis, 5 factors with a total of 16 items extracted that were Social level(Social participation 6 items, Physical Activity 2 items), Physical level(Disease Perception 4 items, Physical Index 2 items), Psychological level(Emotional Support 2 items). The reliability of the AOE health status scale was .868(Conbach's α). In the factor analysis Conbach's α correlations for the 5 factors ranged from .806 to .577. Therefore, Content validity, construct validity, reliability of the AOE health status scale were established. The AOE health status scale is a reliable and valid instrument which abdominal obesity elderly health status can be assessed.

• Therapeutic Science for Rehabilitation
• /
• v.13 no.1
• /
• pp.65-74
• /
• 2024

Objective : This study assessed ChatGPT, an artificial intelligence system based on a large language model, for its ability to pass the National Korean Occupational Therapy Licensure Examination (NKOTLE). Methods : Using NKOTLE questions from 2018 to 2022, provided by the Korea Health and Medical Personnel Examination Institute, this study employed English prompts to determine the accuracy of ChatGPT in providing correct answers. Two researchers independently conducted the entire process, and the average accuracy of both researchers was used to determine whether ChatGPT passed over the 5-year period. The degree of agreement between ChatGPT answers of the two researchers was assessed. Results : ChatGPT passed the 2020 examination but failed to pass the other 4 years' examination. Specifically, its accuracy in questions related to medical regulations ranged from 25% to 57%, whereas its accuracy in other questions exceeded 60%. ChatGPT exhibited a strong agreement between researchers, except for medical regulation questions, and this agreement was significantly correlated with accuracy. Conclusion : There are still limitations to the application of ChatGPT to answer questions influenced by language or culture. Future studies should explore its potential as an educational tool for students majoring in occupational therapy through optimized prompts and continuous learning from the data.

• Journal of Science Education
• /
• v.48 no.1
• /
• pp.31-46
• /
• 2024

This study analyzes the academic achievement characteristics of Korean fourth graders on the problem solving and inquiry tasks (PSIs) introduced in TIMSS 2019. TIMSS 2019 conducted a computer-based assessment in addition to the traditional paper-based assessment. The PSIs were included only in the computer-based assessment, so 30 countries participated in the PSIs of the computer-based assessment. PSIs consist of integrating multiple content and cognitive domains, including 10 or fewer items. Most of the items are constructed in an open-ended format rather than multiple-choice. The analysis results showed that there were differences in student achievement across countries depending on the inclusion of PSIs. Korea's average achievement score decreased by 1 point. The analysis of individual items showed that students' achievement was somewhat low, and the correct answer rate for male students was generally higher than that for female students in many items. Furthermore, item-by-item analysis revealed that there were items where countries such as England and Finland had higher correct answer rates than traditional high-achieving countries, i.e. Singapore, Taiwan, and Korea. Considering the recent emphasis on integrated education, it seems necessary to review the use of PSIs in assessments in Korea as well.

• Journal of The Korean Association For Science Education
• /
• v.31 no.6
• /
• pp.887-900
• /
• 2011

Teachers use explanations to communicate important scientific ideas to students. Consequently, all biology teachers should be evaluated to determine how effective they are at constructing and communicating biological explanations. Open response questions are required to detect pre-service biology teachers' abilities to communicate robust and accurate scientific explanations. Nevertheless, multiple-choice questions are typically preferred by educators because of the common drawbacks of using open-response instruments, such as scoring time, inter-rater scoring disagreements, and delayed feedback to test takers. This study aims to measure pre-service biology teachers' competence in building scientific explanations and to investigate how accurately multiple-choice questions predict the results of open-response questions. One hundred twenty four pre-service biology teachers participated in the study and were administered 20 multiple-choice items and three open-response items designed to measure the accuracy and quality of their explanations of evolutionary change. The results demonstrated that pre-service teachers displayed higher competence when tested with multiple choice items than when tested with open response items. Moreover, scores derived from multiple-choice items poorly predicted the scores derived from open-response items. Multiple-choice items were also found to be poor measures of the consistency, purity and abundance of conceptual elements in teachers' evolutionary explanations. Additionally, many teachers held mixed-models composed of both scientific and naive ideas, which were difficult to detect using multiple-choice formats. Overall, the study indicates that multiple-choice formats are poorly suited to measuring several aspects of biology teachers' knowledge of evolution, including their ability to generate scientific explanations. This study suggests that open-response items should be used in teacher education programs to assess pre-service teachers' explanatory competency prior to being permitted to teach science to children.

• Journal of The Korean Association For Science Education
• /
• v.43 no.1
• /
• pp.17-28
• /
• 2023

In line with the entry into the digital-based intelligent information society, the science curriculum emphasizes the cultivation of scientific competencies, and computer-based test (CBT) is drawing attention for assessment of competencies. CBT has advantages to develop items that have high fidelity, and to establish a feedback system by accumulating results into the database. However, it is necessary to solve the problems of improving validity of assessment results, lowering measurement efficiency, and increasing management factors. To examine students' responses to the introduction of the new assessment tools in the process of transitioning from paper-based test (PBT) to CBT, in this study, we analyzed the results of the PBT and the CBT conducted in 2021 National Assessment of Educational Achievement (NAEA). In particular, we sought to find the effects on student achievement when only the mode of assessment was changed without change of items, and the effect on student achievement when the items were composed including technology enhanced features that take advantage of CBT. This study is derived through the analysis of the results of 7,137 third-grade middle school students taking one among the three kinds of assessments, which were the PBT or two kinds of CBT. After the assessment, the percentage of correct answers and the item discriminations were collected for each group, and expert opinions on characteristics of response were collected through the expert council involving 8 science teachers with experience in NAEA. According to the results, there was no significant difference between students' achievement results in the PBT and the CBT-M, which means simple mode conversion type of CBT, so it could be explained that the mode effect did not appear. However, it was confirmed that the percentage of correct answers for the construct response items was somewhat high in the CBT, and this result was analyzed to be related to the convenience of the response. On the other hand, there were the items with a difference of more than 10%p from the correct answer rate of similar items, among the items to which technology enhanced functions were applied following the introduction of CBT. According to the analysis of response rate of options, these results could be explained that the students' level of understanding could be more closely grasped through the innovative items developed through the technology enhanced function. Based on the results, we discussed some guidance to be considered when introducing CBT and developing items through CBT, and presented implications.

• Journal of The Korean Association For Science Education
• /
• v.24 no.3
• /
• pp.532-543
• /
• 2004

The teacher appointment examination is consisted of a written test, an interview, a demonstration of teaching practices, one's academic record etc. The primary written test(full credit of 100) consists of multiple-choice test of general pedagogy(30%) and constructed response test of science(70%). The science test consists of science education($20{\sim}30%$) and science content($70{\sim}80%$). Five science exams were implemented for appointing general science, physics, chemistry, biology and earth science teachers in 2002 and 2003. The credits alloted to science education domain decreased from $25.7{\sim}30%$ in 2002 to $24.3{\sim}25.7%$ in 2003. The percentage of correct answer was similar between science education and science content domain in 2002, but the percentage of correct answer in science education domain(70.2%) was 29.8% higher than that of science content domain in 2003. Earth science exam almost reached the target mean score of 60 but the other four exams showed the mean scores ranging from 45.0 to 52.7 in 2002 and from 40.1 to 49.6 in 2003. The percentage of high difficulty items(p<40%) was 41.2% in physics and chemistry in 2002, 50% in physics and chemistry, and 45% in biology in 2003. Seventy eight percent of the items showed fairly high discrimination index(r>.30). In addition, the reliabilities of 5 tests were from .79 to .88. In conclusion, it is recommended that the credits alloted to science education domain should be increased up to 30%, and exam difficulties should be lowered to a proper level by making science content items easier.

• Journal of The Korean Association For Science Education
• /
• v.25 no.2
• /
• pp.173-183
• /
• 2005

In this study, we questioned what assessment types and scoring methods of science performance assessment(SPA) were being used in middle and high school, and how much these SPA scores were reliable(generalizable). To answer these questions, SPA data obtained from the seven schools were classified according to assessment type and scoring method. Based upon this classification, we analyzed the reliability by applying generalizability theory. The result, from the classification of assessment type and scoring method, showed that SPA types of the seven schools were divided into two types: paper-pencil type and task type. Paper-pencil type included answer(content)-restricted essay-type test solely. Task type has two parts: process and outcome assessment. As the results of analyzing scoring methods of the seven schools, there were two cases in the way of scoring methods: one case is scoring all essay-type items and performance tasks by one teacher, the other is scoring assigned performance tasks by two teachers. But the case of scoring assigned essay-type items or the case of cross scoring by two or more teachers were not found. The findings of the reliability analysis are as follows: (1) Effect of essay-type item to SPA score was larger than that of performance task. (2) There was remarkable difference among the seven schools' interaction effect of person and rater in scoring performance tasks. (3) Most of generalizability(reliability) coefficients of SPA for the seven schools were smaller than the acceptable generalizability coefficient(0.80). Therefore, the population of statistical parameters such as number of item, task and rater, should be increased for approaching the acceptable generalizability level.

• Journal of The Korean Association For Science Education
• /
• v.20 no.3
• /
• pp.411-420
• /
• 2000

The purpose of this study was to investigate how many students understood the analogies for physics concepts used in middle school science textbooks, and which types of analogies they understood better than others. We classified the analogies into the following 4 types: verbal and simple analogy, verbal and enriched analogy, pictorial and simple analogy, and pictorial and enriched analogy. For the study, 46 students were sampled from a middle school in Ulsan city in Korea, and a tool for testing their understanding of analogies were developed. The tool is composed of 8 items, and its face validity about contents and difficulty was verified by 5 experts. It was found that in average only about 50% of the students understood the analogies in the middle school science textbooks averagely, and that the students understood pictorial and simple analogies better than the other types of analogies.