• Journal of The Korean Association For Science Education
• /
• v.40 no.3
• /
• pp.253-269
• /
• 2020

This study aims to develop items to diagnose pre-service physics teachers' understanding of the conceptual knowledge of modern physics, based on the achievement criteria presented in the 2015 revised national science curriculum, and to identify the validity and reliability of the newly developed items. Data were collected from 467 pre-service physics teachers in the Physical Education Department or Science Education Department (Physics Education Major) of 15 universities across the nation. In this study the content validity, substantive validity, the internal structure validity, generalization validity, and the external validity proposed by Messick (1995) were examined by various statistical tests. The results of the MNSQ analysis showed that there was no nonconformity in the 23 items. The internal structure validity was confirmed by the standardized residual variance analysis, which shows that the 22 items was unidimensional. The generalization validity was confirmed by differential item functioning (DIF) analysis about groups lectured or not modern physics/quantum mechanics. In addition, item analysis and test analysis based on classical test theory were performed. The mean item difficulty is 0.66, mean item discrimination is 0.47 and mean point biserial coefficient obtained was 0.41. These results for item parameters satisfied the criteria respectively. The reliability of the internal consistency of the KR-20 is 0.77 and the Ferguson's delta obtained was δ = 0.972. By Rasch model analysis, the item difficulty (item measures) was discussed.

• The Journal of Korean Academy of Sensory Integration
• /
• v.13 no.2
• /
• pp.31-41
• /
• 2015

Objective : Purpose of this study is to develop a Sensory Processing Scale for Children (SPS-C) to screen sensory processing problems for children aged 3 to 5, and examine validity of the assessment. Method : A draft version of Sensory Processing Scale for Children (SPS-C) has been developed and it was utilized to 138 caregivers of children who are 3 to 5 years old. Rasch analysis was used to test construct validity of SPS-C. Through the Rasch analysis, unidimensionality and adequacy of the rating scale were examined. Results : Based on the Rasch analysis, it was found that six items out of 56 items (10.7%) are misfits. And the results also showed that 3-points rating scale is more adequate than 5-points scale for SPC-C. Conclusion : The construct validity of a newly developed assessment, Sensory Processing Scale for Children (SPS-C), was established through this study. Further studies are needed to examine other psychometric properties of the assessment, using the one modified based on the results in this study.

• Journal of The Korean Association For Science Education
• /
• v.36 no.1
• /
• pp.177-190
• /
• 2016

The purpose of this study is to validate learning progressions for Earth's motion and solar system from two different perspectives of validity. One is construct validity, that is whether a hypothetical pathway derived from our study of LPs is supported by empirical evidence of children's substantive development. The other is consequential validity, which refers to the impact of LP-based adaptive instruction on children's improved learning outcomes. For this purpose, 373 fifth-grade students and 17 teachers from six elementary schools in Seoul, Kangwon province, and Gwangju participated. We designed LP-based adaptive instruction modules delving into the unit of 'Solar system and stars.' We also employed 13 ordered multiple-choice items and analyzed the transitions of children's achievement levels based on the results of pre-test and post-test. For testing construct validity, 64 % of children in the experimental group showed improvement according to the hypothetical pathways. Rasch analysis also supports this results. For testing consequential validity, the analysis of covariance between experimental and control groups revealed that the improvement of experimental group is significantly higher than the control group (F=30.819, p=0.000), and positive transitions of children's achievement level in the experimental group are more dominant than in the control group. In addition, the findings of applying Rasch model reveal that the improvement of students' ability in the experimental group is significantly higher than that of the control group (F=11.632, p=0.001).

• Journal of The Korean Association For Science Education
• /
• v.42 no.2
• /
• pp.239-252
• /
• 2022

This study aims to develop performance criteria based on characteristics observed in science inquiry tasks for elementary school students. First, the performance characteristics by observing 70 fifth-grade elementary school students' science inquiry activity report are listed. Second, the checklist-type scoring criteria in connection with the theoretical framework of scientific inquiry process and relevant competencies are developed. Third, with the developed scoring criteria, 11 raters participate in scoring 350 students' reports. The main findings are as follow: first, the scoring data are well-fitted for the many-faceted Rasch model, and 22 scoring criteria are reasonably-well differentiated for various levels of proficiency. Second, at low performance level, observable characteristics are to answer questions explicitly required by the task or to observe objects or phenomena using pre-learned scientific concepts, while at high performance level, to explore additional data other than given data or to reflect on one's experimental process. Based on the results, the usefulness of analyzing students' performance characteristics for developing the scoring criteria, and further research directions are discussed.

• Journal of Science Education
• /
• v.41 no.3
• /
• pp.382-404
• /
• 2017

This study aims to explore a learning progression for eight core concepts of middle school science using constructed response items in the national assessment of educational achievement (NAEA). For this purpose, a total of 7,390 responses of the 9th graders on 8 constructed response items in NAEA conducted in 2015 were inputted by computer and the inputted answers were analyzed and classified into different levels. After completing this process, five levels were set for each construct and a construct map was created according to the levels. In this study, the construct map was written in a progression-based rubric format, which was used as a criterion for leveling the answers of all students. In addition, the Rasch model was applied to measure the scores and achievements of the respondents by means of statistical analysis to correlate the scores of the students with the achievement levels of the constructs. Lastly, a preliminary learning progression was created by revising the construct map reflecting the results of Rasch model application. This study was meaningful in that it explored the possibility of developing the learning process by using constructed response items in NAEA. However, the preliminary learning progression developed in this study is still hypothetical and inferential because it is not longitudinally traced to individual students. Therefore, it is necessary to continually revise and supplement through iterative research process.

• Journal of Korean Critical Care Nursing
• /
• v.15 no.3
• /
• pp.37-50
• /
• 2022

Purpose : This study aimed to examine the item characteristics of the Korean version of the intensive care experience questionnaire (K-ICEQ) using the Rasch analysis model of the item response theory. Methods : In this methodological study, the validity of the scale was examined, and a secondary analysis was conducted using cohort data of patients who were discharged from the intensive care units (ICU). Data from 891 patients who responded to the K-ICEQ upon ICU discharge were analyzed. The WINSTEP program was used to analyze item characteristics, including item difficulty, fit indices, appropriateness scale, and separation reliability. Results : The difficulty level of all 26 items of the K-ICEQ was appropriate, and the fit indices of the 25 items, except for item 18, were good. The 5-point scale of the K-ICEQ was not appropriate in the three subscales. The item separation reliability was good in all subscales, but did not meet the criteria in terms of respondents. Conclusion : The results of examining the item characteristics of the K-ICEQ revealed a good degree of difficulty, fitness, and item separation reliability. To increase the validity of the K-ICEQ, we suggest the rearrangement of the overall item order, modification of the item description of the "recall of experience" subscale, and reduction of the scale response level.

• Journal of Science Education
• /
• v.41 no.1
• /
• pp.16-35
• /
• 2017

This study attempted to investigate the relationship among engineering students' creativity, engineering education accreditation, gender, and academic year. To be specific, fist, we examined the validity and reliability of the instrument for measuring engineering students' creativity by conducting Rasch analysis. Second, we compared level of creativity in terms of gender, academic year, and engineering education accreditation by using three-way ANOVA. Third, correlations among four constructs of creativity were examined. Fourth, latent classes with respect to creativity within the participants were identified using polytomous latent class analysis (poLCA). Data were collected from 2098 engineering students by using instrument for measuring four different constructs (creative cognition, creative tendency, collaboration, environment). By using Rasch analysis, validity and reliability of instrument for measuring creativity were confirmed. And the results of three-way ANOVA showed that there were significant difference in creativity in terms of gender. Female students showed the low level of creativity compared with male students. Also there were significant difference in creativity except creative cognition factor in terms of academic year. But there were no significant difference in creativity between students who participated engineering education accreditation program and the others. All constructs of creativity were significantly correlated with each others. Lastly, poLCA results showed that there were three distinct subgroups within engineering students in terms of the level of creativity. In the subgroup with low creativity, there were more female and first year students. Based on these findings, we discussed education for engineering students' creativity.

• 재활복지
• /
• v.18 no.4
• /
• pp.315-330
• /
• 2014

The purpose of this study was to verify the construct validity of the Sensory Profile for children with congenital cerebral palsy. Parents of 87 children(the ages of 3 to 10) with congenital cerebral palsy participated in this study. The data were analyzed through Winstep version 3.81 using the Rasch model to examine the uni-dimensionality of the fit of each item, the distribution of difficulty of each item, and the reliability and appropriateness of the rating scale. Based on a Rasch analysis, four out of the 87 children were considered to be inappropriate participants, and 15 item of the Sensory Profile was determined to be an inappropriate item. The items of high-level difficulty are needed as new items of the Korean Sensory Profile. The rating scale of three categories were appropriate than those of the five categories. The person and item separation reliability of three categories was above 0.90, which is a relatively excellent value. Finally, it will be need to verify of validity for Korean version of Sensory Profile, to develop new item of a high level of difficulty and convert into three point rating scale.

• Communications of Mathematical Education
• /
• v.18 no.2 s.19
• /
• pp.239-250
• /
• 2004

수학적 힘의 함양과 문제해결력의 신장을 위한 수학교육에서 확률영역은 중요한 학습소재임에도 불구하고, 확률영역은 어려운 것으로 고착되었다. 이 연구에서는 학생들이 확률영역의 어떤 부분을 어려워하고 이해하기 힘들어하는지를 구체적 문항분석을 통하여 알아봄으로서 교수-학습의 기초자료를 제공하고자한다. 이를 위하여, 지난 10년간 출제되었던 대학수학능력시험의 확률영역 16문항을 고등학교 학생 220명에게 실시하고, 고전검사이론과 문항반응이론울 적용하여 그 결과를 분석하였다. 고전검사이론에서는 신뢰도와 변별도를 측정하였고, 문항반응이론에서는 Rasch 1-모수 문항반응모형에 근거한 BIGSTEP을 사용하여 내적타당도와 난이도를 측정하였다.

• Journal of The Korean Association For Science Education
• /
• v.36 no.3
• /
• pp.509-517
• /
• 2016

Previous studies have investigated the similarity between the development of evolutionary explanations and students' conceptual developments on evolution. However, the validity and reliability of the assessment method reflecting the similarity have not been quantitatively examined yet. In addition, no study has examined the conceptual progressions of evolution concept based on contexts although literature has addressed the contextual difference of evolutionary explanation in the history of science. This study examined the validity and reliability of history-of-science-based evolution concept assessment using ordered multiple choice (OMC) methods and Rasch analysis and explored conceptual progression by three contexts (e.g., human, animal, and plant). The evolution concept assessment developed by Ha (2007) was used to examine 1711 elementary, middle, and high school students, and pre- and in-service science teachers' (biology majors and non-majors) evolution concepts. Internal consistency reliability and item response fitness of the OMC method that provide 0- to 4-point scores to creationism, teleology, intentionality, use/disuse, and natural selection respectively met the benchmark based on the Cronbach alpha and MNSQ indices of Rasch analysis. The level of elementary and middle school students' evolution concepts were located between intentionality and use/disuse while the level of high school and non-biology science teachers' evolution concepts were located between use/disuse and natural selection. The conceptual progressions of evolution concepts were differentiated according to three contexts. This study provided the quantitative evidence for the similarity between the development of evolutionary explanations and students' conceptual developments on evolution and suggest new analysis methods (i.e., OMC) of evolution concept assessment.