• Journal of Engineering Education Research
• /
• v.26 no.6
• /
• pp.40-50
• /
• 2023

This study is to examine the validity of assessing basic-level proficiency in physics among incoming engineering freshmen through item analysis and standard setting. For empirical analysis, we examined the physics subject taken by the freshman class of 2021 at K University, considering its significance for engineering students. In this study, we initially performed item analysis utilizing both classical test theory and item response theory. Subsequently, leveraging the item and test information, we employed a modified Angoff method and the Bookmark method for standard setting. Consequently, the difficulty level initially set during item development was found to be higher than the actual performance level exhibited by the students. This study highlights a discernible disparity between the expected university standard and the real proficiency level of incoming freshmen in terms of basic academic ability in physics. Based on these research findings, a comprehensive discussion on the fundamental academic competence of engineering students was conducted, underscoring the necessity for formulating a tailored learning approach leveraging the outcomes from the basic ability test.

• Journal of Science Education
• /
• v.37 no.2
• /
• pp.278-293
• /
• 2013

The purpose of this study was to analyze the characteristics of multiple-choice test items used in assessment of high school integrated science according to 2009 revised curriculum. For the analysis of the tendency of item setting, we devised an analytic framework specific to integrated science, and analyzed the characteristics of items by applying the devised framework and item response theory. The results of the tendency of item setting revealed that most of items run counter to the intent of integrated science in terms of item resource, integration extent, and cognitive level, which means teachers are stick to separative method in teaching-learning and assessment of integrated science. The results of the analysis applying item response theory showed that item difficulty was appropriate and item discrimination was considerably high. However, there was no relevance between the tendency of item setting and qualitative characteristics of the items. We also discussed some agendas to improve the teaching-learning and assessment of integrated science based on the results of this study.

• Journal of the Korean earth science society
• /
• v.24 no.3
• /
• pp.150-159
• /
• 2003

This study suggests methods of assessing scientific creativity and developing items, which can be achieved when both earth science knowledge and general creativity are applied at the same time. According to the results of this study, the cognitive ability gaps between creativity and scientific creativity were clearly defined by the terms' operational definition. Four factors in the Subcategory Of Scientific Creativity-fluency, flexibility, elaboration, and originality-were selected, and the possibility of developing items out of these factors was discovered. The operational definitions of the four factors were given and the criteria for assessment and scoring were set. The validity, reliability, discrimination, and difficulty, which were the conditions required for the assessment instruments, were verified through three field trials of inputting the assessment instruments for scientific creativity. The assessment instruments were composed of 8 items with 2items for each factor. The average item fitness index obtained was 0.99, Cronbach , the item inter-consistency was 0.79,the inter-rater reliability of each item was 0.78, the inter-rater reliability of each factor was 0.75, the item discrimination power was 0.19, and the item difficulty was 0.00. Because the results were within the permitted limit of the conditions required for assessment instruments, the assessment instruments developed for scientific creativity in this study can be said to be very favorable.

• The Journal of Korean Academy of Sensory Integration
• /
• v.18 no.2
• /
• pp.15-28
• /
• 2020

Objective : The purpose of this study was to verify the construct validity of Short Sensory Profile-2 (SSP-2) for children with Autism Spectrum Disorder (ASD). Methods : Data were collected from SSP-2 for 120 parents of ASD children. Raw data were analyzed by applying the Rasch analysis to the goodness fit of person and item, item difficulty, rating scale, and separation reliability of SSP-2. Results : 7 persons in sensory processing area and 8 persons in behavioral response area were inappropriate criteria and excluded from the analysis. Item goodness-of-fit analysis determined that the If the Mnsq value is between 0.6 and 1.4 and the Z value is outside the ±2 range for nonconformity. this study All items in the instrument were found to have appropriate criteria. Item difficulty analysis in sensory processing area was high in item 13 (.48 logit) and low in item 10 (-.54 logit). In the behavioral response area, item 25 (1.58 logit) was high and item 30 (-.68 logit) was low. In the rating scale analysis, it was found that the 3-point scale is more appropriate than the 5-point scale. The separation reliability of sensory processing area was .90 and the behavioral response area was .95. Conclusion : This study verified the construct validity of SSP-2 and expected to be applied as a useful evaluation tool for children with ASD.

• Journal of Preventive Medicine and Public Health
• /
• v.35 no.4
• /
• pp.359-374
• /
• 2002

Objectives : To develop a physical functioning instrument for older adults living in the community. Methods : A representative sample of 979 people aged 65 years or over were interviewed in-person. Of these, 199 people also completed a detailed in-hospital examination. The scale items were selected based on the frequency of endorsement, along with the item-total and inter-item correlations. The associations of the scale with their physical performance and clinical examination were analyzed to evaluate the criterion-related validity. Construct validity was assessed using factor analysis, and internal consistency through Cronbach's alpha and item-total correlations. Test-retest reliability was measured by agreement between the household survey and the repeat survey at the in-hospital examination. Results : Initially, 23 items on the level of difficulty, ranging from no difficulty to an inability to complete a task, with the specific mobility and self-care tasks were included. Those with a high frequency of endorsement and a low inter-item or item-total correlations were excluded, resulting in a 10-item Physical Functioning (PT) scale. Equal weights were given to each item and a summated score was calculated. Significant associations were found between the PF scores and the physical performance, surrey and clinical data. The scale revealed a 2-factor (mobility and self-care) structure. Cronbach's alpha was 0.92 and the item-total correlations were in the 0.63 to 0.78 range. Pearson's correlations for the test-retest ranged between 0.56 and 0.61. Conclusions : The newly developed Physical Functioning (PF) scale showed good psychometric properties in older people. Further work, however, is needed to improve its sensitivity to discriminate higher levels of functioning, in addition to assessing its predictive value in detecting changes in health.

• Journal of Korea Society of Industrial Information Systems
• /
• v.9 no.4
• /
• pp.47-55
• /
• 2004

Most questions made for remote examinations on web-based education system use methods of making questions using fixed questions or randomly using item pools or automatically using degree of difficulty. Particularly, automatically selection methods using degree of difficulty is the kernel of a question that objectivity of examination questions by degree of difficulty adjustment based result of examination. This paper is use automatically selection methods for examination on web-based education system. Therefore we present new algorithms of mediateness degree of difficulty as regards education ability of students for adjust the degree of difficulty. We identified this algorithms is more effective as compared with previously algorithms on web-based education system

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.15 no.1
• /
• pp.203-208
• /
• 2011

Most of the online learning systems provide information that is based on the item difficulty what is custumized to learner. And the same learning process is performing to the same learning level learners. But, the degree of understanding of the same learning contents can be different even if the learner's level is same. Therefore, it is difficult to represent an effective learning experience because the learning is progressed by the determined difficulty of learning and the learning process even thought the provided content is difficult to understand. So we can control the learning difficulty during learning in order to escape from a uniform learning that online learning is provided. In this paper, we proposed a contents reorganization method for Self-Directed Learning. In this way, learners can understand their own level and customize the difficulty of learning. And then we expect the higher learning effect and satisfaction degree.

• Journal of Science Education
• /
• v.34 no.1
• /
• pp.155-163
• /
• 2010

The purpose of this study was to analyze the items that were used in entrance examination for science gifted education center for elementary school students by using content analysis and classical item analysis. In content analysis, objective type items exhibited matter and interpreting data were dominant. And essay type items consisted of creativity items, evaluated creative problem solving ability. Item difficulty and discrimination index, on the whole, were appropriate. Comparing with objective type, essay type has higher discrimination index. In correlation analysis between total score and score of each type of items, total score has the highest correlation with matter items, interpreting data and creativity.

• Journal of The Korean Association For Science Education
• /
• v.40 no.3
• /
• pp.253-269
• /
• 2020

This study aims to develop items to diagnose pre-service physics teachers' understanding of the conceptual knowledge of modern physics, based on the achievement criteria presented in the 2015 revised national science curriculum, and to identify the validity and reliability of the newly developed items. Data were collected from 467 pre-service physics teachers in the Physical Education Department or Science Education Department (Physics Education Major) of 15 universities across the nation. In this study the content validity, substantive validity, the internal structure validity, generalization validity, and the external validity proposed by Messick (1995) were examined by various statistical tests. The results of the MNSQ analysis showed that there was no nonconformity in the 23 items. The internal structure validity was confirmed by the standardized residual variance analysis, which shows that the 22 items was unidimensional. The generalization validity was confirmed by differential item functioning (DIF) analysis about groups lectured or not modern physics/quantum mechanics. In addition, item analysis and test analysis based on classical test theory were performed. The mean item difficulty is 0.66, mean item discrimination is 0.47 and mean point biserial coefficient obtained was 0.41. These results for item parameters satisfied the criteria respectively. The reliability of the internal consistency of the KR-20 is 0.77 and the Ferguson's delta obtained was δ = 0.972. By Rasch model analysis, the item difficulty (item measures) was discussed.

• Korean Journal of Biological Psychiatry
• /
• v.20 no.2
• /
• pp.40-44
• /
• 2013

Objectives The purpose of this study was to examine the differences according to depressive symptom-related difficulty status. Methods 2828 participants were a divided into depressive symptom-related difficulty group (difficult group, n = 774), and a non-depressive symptom-related difficulty group (not difficult group, n = 2054). The psychological character of the participants were assessed using the Korean version of the Patient Health Questionnaire-9 (PHQ-9), Satisfaction with Life Scale (SWLS), the 12-item General Health Questionnaire (GHQ-12), and Conner-Davidson Resilience Scale (CD-RISC). Statistical analyses were done using t-test, chi-square, and analysis of covariance (ANCOVA). Results Compared with the no difficulty group, the difficulty group reported significantly higher score in all items of PHQ-9. The score of "feeling tired" was the highest and the score of "suicidal ideation" is the lowest in both groups. ANCOVA analysis that is adjusted with the total score of PHQ-9 showed the differences in SWLS, GHQ-12, and CD-RISC scores between the difficulty group and the no difficulty group. Conclusions The findings suggest that there are different characters on PHQ-9, SWLS, GHQ-12, and CD-RISC according to depressive symptom-related difficulty. Therefore, it is required not only to evaluate depressive symptoms in patients with depression, but also the depressive symptom-related difficulty to understand these differences.