• Journal of Convergence for Information Technology
• /
• v.10 no.3
• /
• pp.115-125
• /
• 2020

Problem-Based Learning is the most commonly used teaching method in the field of education as the demand for talented people with creative problem-solving abilities has increased in recent society. In this study, we developed a tool that can simply and regularly measure the learning effects of applying Problem-Based Learning on the classroom. Twenty-two preliminary items for the development were constructed by gathering expert opinions about items collected in existing research of Problem-Based Learning effects measurement. Using this preliminary items, data were collected from 124 students who took the subject-based study. And the validity and reliability were verified through Rash analysis. 11 of 124 students and 1 of 22 items were found to nonconforming and excluded. In addition, the optimal scale for the configuration of this tool was found to be a four-point scale, and the reliability of separation on the subject and the item was found to be excellent. We hope that the assessment tools of Problem-Based Learning effectiveness developed in this study will be actively used to measure learning effectiveness and manage the quality of education.

• Korean Journal of Culture and Social Issue
• /
• v.30 no.2
• /
• pp.159-192
• /
• 2024

The purpose of this study is to validate the Reactive-Proactive Aggression Questionnaire (RPQ), a tool for measuring reactive-proactive aggression, in the context of South Korea. A thorough translation was conducted in collaboration with the original author. An exploratory factor analysis (EFA), exploratory structural equation modeling (ESEM), rating scale model (Rasch), differential item functioning (DIF), and convergent validity were performed on a sample of 510 South Korean individuals. The results revealed a two-factor structure of reactive and proactive aggression after removing one item showing dual loading. Rating scale analysis based on the Rasch model indicated the appropriateness of the 3-point Likert scale, with all items meeting fit criteria. Although the separation index and separation reliability of proactive aggression was marginally lower, the overall discrimination between participants and items was satisfactory. Examination of participant-item distribution indicated a suitable alignment between reactive aggression and participant ability levels, whereas proactive aggression exhibited slightly elevated item difficulty. Furthermore, three items were found to function differently based on gender. A moderate but statistically significant positive correlation was found between the Barratt Impulsiveness Scale-11-R (Korean version) and RPQ from the results of the convergent validity evaluation. Overall, this study employed rigorous statistical methods to validate the suitability of the RPQ for use in Korea, taking cultural nuances into account, and introduced the concepts of reactive and proactive aggression to the Korean general population.

• English Language & Literature Teaching
• /
• v.18 no.2
• /
• pp.125-147
• /
• 2012

The primary goal of the study was to provide some preliminary validity evidence for the Vocabulary Grade Levels Test (Busan Metropolitan City Office of Education, 2009), which is designed to measure the receptive vocabulary knowledge of learners in L2. For the purpose of the current study, 327 participants at the elementary school participated in the study and were asked to take two different vocabulary tests. Namely, a Vocabulary Size Test (Nation, 2001) and a Vocabulary Grade Levels Test. The data were analyzed using correlation in order to discover the relationship between these two types of tests. Following this, the Rasch analysis was conducted to examine the reliability and validity of the measurement in question. The data analysis showed that both grade separation reliability and item separation reliability were high, indicating that the Vocabulary Grade Levels Test well discriminates learners with a wide range of proficiency levels. The findings of the study are discussed, along with further improvements in order to ascertain the validity of this particular vocabulary test.

• International Journal of Contents
• /
• v.15 no.3
• /
• pp.1-6
• /
• 2019

The purpose of this study was to identify the psychometric properties of the vocational ability scale used in the 8th Panel Survey of Employment for the Disabled in Korea by using the Rasch model. The sample data was collected from 398 individuals with intellectual disabilities. Item fitness, item difficulty, the appropriateness of the rating scale, and the separation index of the vocational ability scale were evaluated. All 15 items show an appropriate fitness level. The analysis of item difficulty indicate that modifications are required. Specifically, the need for the addition of less difficult question items is identified. The use of a 5-point rating scale is shown to decrease the test difficulty in terms of clarity and readability when appropriate and a 4-point modification is also determined as appropriate. With respect to the outcomes of the analysis, the person separation reliability value and separation index are high, and the reliability of the items is also high.

• Journal of The Korean Association For Science Education
• /
• v.40 no.3
• /
• pp.253-269
• /
• 2020

This study aims to develop items to diagnose pre-service physics teachers' understanding of the conceptual knowledge of modern physics, based on the achievement criteria presented in the 2015 revised national science curriculum, and to identify the validity and reliability of the newly developed items. Data were collected from 467 pre-service physics teachers in the Physical Education Department or Science Education Department (Physics Education Major) of 15 universities across the nation. In this study the content validity, substantive validity, the internal structure validity, generalization validity, and the external validity proposed by Messick (1995) were examined by various statistical tests. The results of the MNSQ analysis showed that there was no nonconformity in the 23 items. The internal structure validity was confirmed by the standardized residual variance analysis, which shows that the 22 items was unidimensional. The generalization validity was confirmed by differential item functioning (DIF) analysis about groups lectured or not modern physics/quantum mechanics. In addition, item analysis and test analysis based on classical test theory were performed. The mean item difficulty is 0.66, mean item discrimination is 0.47 and mean point biserial coefficient obtained was 0.41. These results for item parameters satisfied the criteria respectively. The reliability of the internal consistency of the KR-20 is 0.77 and the Ferguson's delta obtained was δ = 0.972. By Rasch model analysis, the item difficulty (item measures) was discussed.

• Physical Therapy Korea
• /
• v.15 no.2
• /
• pp.44-53
• /
• 2008

The purpose of this preliminary study was to develop a measurement for assessing risk factors for falling in community-dwelling elderly persons. Rasch analysis and principal component analysis were performed to examine whether items on the Activities-Specific Balance Confidence (ABC), assessing self-efficacy, and items on the Berg Balance Scale (BBS), assessing balance function, contribute jointly to a unidimensional construct in the elderly. A total of 35 elderly persons (4 men, 31 women) participated. In this study, each item of ABC (16 items) and BBS (14 items) was scored on a 5-point ordinal rating scale from 0 to 4. The initial Rasch and principal component analysis indicated that 3 of the ABC items and 2 of the BBS items were misfit for this study. These 5 items were excluded from further study. After combining ABC and BBS, Rasch and principal component analyses were examined and finally 23 items selected; 12 items from ABC, 11 items from BBS. The 23 combined ABC-BBC items were arranged in order of difficulty. The hardest item was 'walk outside on icy sidewalks' and the easiest item was 'pivot transfer'. Although structural calibration of each 5 rating scale categories was not ordered, the other three essential criteria of Linacre's optimal rating scale were satisfied. Overall, the ABC-BBS showed sound item psychometric properties. Each of the 5 rating scale categories appeared to distinctly identify subjects at different ability levels. The findings of this study support that the new ABC-BBS scale measure balance function and self-efficacy. It will be a clinically useful assessment of risk factors for falling in the elderly. However, the number of subjects was too small to generalize our results. Further study is needed to develop a new assessment considering more risk factors of falling in elderly.

• Journal of Korean Critical Care Nursing
• /
• v.15 no.3
• /
• pp.37-50
• /
• 2022

Purpose : This study aimed to examine the item characteristics of the Korean version of the intensive care experience questionnaire (K-ICEQ) using the Rasch analysis model of the item response theory. Methods : In this methodological study, the validity of the scale was examined, and a secondary analysis was conducted using cohort data of patients who were discharged from the intensive care units (ICU). Data from 891 patients who responded to the K-ICEQ upon ICU discharge were analyzed. The WINSTEP program was used to analyze item characteristics, including item difficulty, fit indices, appropriateness scale, and separation reliability. Results : The difficulty level of all 26 items of the K-ICEQ was appropriate, and the fit indices of the 25 items, except for item 18, were good. The 5-point scale of the K-ICEQ was not appropriate in the three subscales. The item separation reliability was good in all subscales, but did not meet the criteria in terms of respondents. Conclusion : The results of examining the item characteristics of the K-ICEQ revealed a good degree of difficulty, fitness, and item separation reliability. To increase the validity of the K-ICEQ, we suggest the rearrangement of the overall item order, modification of the item description of the "recall of experience" subscale, and reduction of the scale response level.

• The Journal of Korean Academy of Sensory Integration
• /
• v.13 no.2
• /
• pp.31-41
• /
• 2015

Objective : Purpose of this study is to develop a Sensory Processing Scale for Children (SPS-C) to screen sensory processing problems for children aged 3 to 5, and examine validity of the assessment. Method : A draft version of Sensory Processing Scale for Children (SPS-C) has been developed and it was utilized to 138 caregivers of children who are 3 to 5 years old. Rasch analysis was used to test construct validity of SPS-C. Through the Rasch analysis, unidimensionality and adequacy of the rating scale were examined. Results : Based on the Rasch analysis, it was found that six items out of 56 items (10.7%) are misfits. And the results also showed that 3-points rating scale is more adequate than 5-points scale for SPC-C. Conclusion : The construct validity of a newly developed assessment, Sensory Processing Scale for Children (SPS-C), was established through this study. Further studies are needed to examine other psychometric properties of the assessment, using the one modified based on the results in this study.

• The Journal of the Korea Contents Association
• /
• v.15 no.10
• /
• pp.354-361
• /
• 2015

The purpose of this study is to investigate validation of the Recovery Assessment Scale(RAS) for Korean Version using Rasch analysis. This study included 60 patients who had chronic schizophrenia and were aged 18 or older. The data were analysed using a Rasch analysis to investigate whether the persons and items fit, the distribution of item difficulty and the appropriateness of the rating scale. 1 out of 60 subjects were found to not fit. 3 out of 24 items were also found to be misfit. The misfitted items was 'I believe that I can meet my current personal goals','Even when I don't care about myself, other people do. ','Coping with my mental illness is no longer the main focus of my life.'. Of the appropriate items, the most difficult was 'I can handle what happens in my life' and the easiest was 'Something good will eventually happen'. The RAS rating scale was shown to be appropriate for patients who had chronic schizophrenia. The item separation reliability was .70, and the person separation reliability was .94. Further studies are needed to investigate validity and criterion cut-off score for many patients with chronic mental illess.

• Journal of Science Education
• /
• v.41 no.3
• /
• pp.382-404
• /
• 2017

This study aims to explore a learning progression for eight core concepts of middle school science using constructed response items in the national assessment of educational achievement (NAEA). For this purpose, a total of 7,390 responses of the 9th graders on 8 constructed response items in NAEA conducted in 2015 were inputted by computer and the inputted answers were analyzed and classified into different levels. After completing this process, five levels were set for each construct and a construct map was created according to the levels. In this study, the construct map was written in a progression-based rubric format, which was used as a criterion for leveling the answers of all students. In addition, the Rasch model was applied to measure the scores and achievements of the respondents by means of statistical analysis to correlate the scores of the students with the achievement levels of the constructs. Lastly, a preliminary learning progression was created by revising the construct map reflecting the results of Rasch model application. This study was meaningful in that it explored the possibility of developing the learning process by using constructed response items in NAEA. However, the preliminary learning progression developed in this study is still hypothetical and inferential because it is not longitudinally traced to individual students. Therefore, it is necessary to continually revise and supplement through iterative research process.