• Title/Summary/Keyword: Consistency in Assessment

Search Result 333, Processing Time 0.022 seconds

Reliability and Validity of the Korean Version of Melbourne Low-Vision ADL Index (한국판 맬버른 저시력 일상생활지수(Melbourne Low-vision ADL Index: MLVAI)의 신뢰도 및 타당도)

  • Yoo, Yeon Hwan;Park, Ji-Hyuk;Jung, Min-Ye;Park, Hae Yean
    • Therapeutic Science for Rehabilitation
    • /
    • v.7 no.2
    • /
    • pp.51-61
    • /
    • 2018
  • Objective: The purpose of this study was performed to modify the performance-based assessment tool, Melbourne Low-Vision ADL Index (MLVAI) had been developed in Australia to suit the Korea culture to verify the reliability and validity. Methods: The subjects were only over the age of 20 living in communities, 26 with low-vision and 42 normal persons. The Korean MLVAI was completed through the expert translation verification and validation of the configuration tool. The validity of the Korean MLVAI was established through the content, discriminant, and convergent validity. Also, the reliability was analyzed through internal consistency reliability for the items, test-retest, and interrater reliability. Results: The Content Validity Index(CVI) was more than .78. There was a statically significant low in low-vision. Also, there was a statically significant low in low-vision. The convergent validity was determined the correlation coefficient of .751 analyzing LVQOL and Korean MLVAI total score, had a significant correlation(p<.05). Cronbach's ${\alpha}$ coefficient indicated an internal consistency of .983(p<.05). Test-retest reliability had a high, significant correlation of .976 and interrater reliability had a high, an intraclass correlation coefficient of .91(p<.05). Conclusion: The results of this study mean that the Korean MLVAI which was modified to fit the Korean is the ADL assessment tool with both validity and reliability. Based on this study, the Korean MLVAI can be used as a useful ADL assessment for OT interventions in low vision.

Development and Validation of the Cancer-Specific Posttraumatic Growth Inventory (암환자 특이형 외상후 성장 측정도구 개발 및 타당화)

  • Jung, Young-Mi;Park, Jin-Hee
    • Journal of Korean Academy of Nursing
    • /
    • v.47 no.3
    • /
    • pp.319-331
    • /
    • 2017
  • Purpose: The purpose of this study was to develop a scale to evaluate posttraumatic growth in patients with cancer and to examine the validity and reliability of the scale. Methods: A literature review, semi-structured patient interviews and an expert panel consultation produced a 27 preliminary item questionnaire. Participants were 150 cancer patients recruited to test the reliability and validity of the preliminary scale. Data were analyzed using item analysis, exploratory factor analysis, convergent validity and internal consistency. Results: Item reduction and exploratory factor analysis led to 23 items, grouped into five subscales which were labelled new possibilities (6 items), coping skills (5 items), preciousness of life (5 items), relating to others (4 items), and personal strength (3 items). Convergent validity was evaluated by total correlation with the Functional Assessment of Cancer Therapy-General (r=.45, p<.001). The final scale demonstrated satisfactory internal consistency (Cronbach's ${\alpha}$ =.94). Conclusion: Findings from this study indicate that the Cancer-Specific Posttraumatic Growth Inventory has validity and reliability and is considered to be appropriate for assessing posttraumatic growth in patients with cancer.

The study of development on CFT(Cognitive Function Test) in mild cognitive impairment (경도인지장애 대상자의 인지기능평가도구(CFT) 개발 연구)

  • Lee, Seon-Myeong;Ju, Eun-U;Song, Jin-Seok;Park, Sin-Jun;Kim, Heon-Mo;Jeong, Eun-Seung;Im, Chae-Ho;Son, Seong-Min;Ha, Hye-Jeong
    • Journal of Korean Clinical Health Science
    • /
    • v.8 no.2
    • /
    • pp.1426-1435
    • /
    • 2020
  • Objective: This study aims to develop CFT, a screening test for dementia that can be conducted on patients with mild cognitive impairment or early dementia. Methods: This study was conducted on therapists working at occupational therapy hospital in Gyeongsangnam-do. The data collection period is two months from July to September 2020, and the research participants were briefed on the purpose, significance, and anonymity of the research in consideration of ethical issues. A total of 33 copies of the data were finally analyzed, excluding those containing insufficient answers and errors, by distributing 40 copies of the data. SPSS Data Access pack 8.0 was used for data analysis, Cronbach's α value was evaluated for internal consistency reliability of each item, and Pearson Correlation coeffient between the lower scales of the assessment tool was measured. Results: The Cronbach's α value of the CFT was shown as .892 and the reliability of the assessment paper was shown to be quite high. The Cronbach's α value for each item in the cognitive assessment tool was the highest at .899 in the subtraction item, with exercise control (mook Chi Pa) at 888, visual memory, plus .887. The correlation of each item of cognitive assessment tools was found to be most correlated with the number of sustained concentrates and word2 to .934; in the stop word-numeric item and visual memory to be .85; and in the addition to the auditory memory item .739. Conclusion: Therefore, this study developed cognitive evaluation tools that can distinguish the diversity and cognitive level of screening tests and clarify standards, which are different from the existing dementia screening tools in Korea. Furthermore, the cognitive assessment tools of this study can be verified by applying them to patients with mild cognitive impairment and early dementia, and the criteria for generalization can be established.

Exploration on the Feasibility of Utilization and Teacher Perceptions of Using ChatGPT for Student Assessment in Science (과학 교과의 학생 평가에서 ChatGPT의 활용 가능성 및 교사 인식 탐색)

  • Dongwon Lee;Hyeon-Pyo Shim;Jongho Baek
    • Journal of The Korean Association For Science Education
    • /
    • v.44 no.1
    • /
    • pp.119-130
    • /
    • 2024
  • This study explores the possibility of using a generative artificial intelligence, ChatGPT, for student assessment in science subjects. In order to achieve our goal, we developed assessment items, collected students' responses, and input them into ChatGPT to implement the assessment procedures. Subsequently, we shared the assessment results from ChatGPT with science teachers and compared them to the teachers' assessment process to investigate the use of ChatGPT in student assessment. Regarding the results, in terms of setting the scoring rubric, we found the rubric generated by ChatGPT to be generally appropriate. However, the consistency between the scoring results obtained from ChatGPT and those determined by the teachers was relatively low. This inconsistency was more pronounced in items with additional assessment components and a more intricate rubric. In regard to feedback on student responses, there were some instances where the feedback generated was scientifically incorrect or beyond the scope of the curriculum, but there were also some positives, such as the provision of exemplary answers to questions and additional examples that helped students learn further. From these results, the teachers perceived limitations in using ChatGPT to conduct assessment in terms of reliability, which is considered crucial in student assessment, but suggested that it could be used to support assessment. Finally, synthesizing these findings, implications for utilizing ChatGPT in student assessment were suggested.

The Validity and Reliability of the Second Korean Working Conditions Survey

  • Kim, Young Sun;Rhee, Kyung Yong;Oh, Min Jung;Park, Jungsun
    • Safety and Health at Work
    • /
    • v.4 no.2
    • /
    • pp.111-116
    • /
    • 2013
  • Background: The aim of this study was to evaluate the quality of the Second Korean Working Conditions Survey (KWCS), focusing on its validity and reliability. Methods: The external validity was evaluated by the assessment of sampling procedures and the response rate, in order to investigate the representativeness of the sample. The content validity was evaluated by the assessment of the development of the questionnaire, and the consistency of questions for the selected construct. The test-retest method was used to evaluate the reliability by means of a phone call survey of 30% of the respondents, who were randomly selected. The respondents' satisfaction regarding the survey procedures and interview time were analyzed to evaluate the quality of survey data. Results: The external validity was assured by an acceptable sampling procedure, rigid multi-stage stratified cluster random sampling. The content validity was also guaranteed by a reasonable procedure for the development of the questionnaire with a pretest. The internal consistency of the questions for work autonomy was maintained, with 0.738 of Cronbach's alpha. The response rate of 36% was lower than that of the European Working Conditions Survey (EWCS), with a contact rate of 66%, compared to 76% for the EWCS. The matching rates of the five retested questions were more than 98% reliable. Conclusion: The quality of the second KWCS was assured by the high external and content validity and reliability. The rigid sampling procedure and development of the questionnaire contributed to quality assurance. The high level of reliability may be guaranteed by the sophisticated field survey procedures and the development of a technical manual for interviewers. The technical strategies for a high response rate should be developed for future surveys.

Development and Standardization of Modified Self-Assessment Manikin for Emotional Valence and Arousal Manikin (정서가 및 각성수준에 대한 자가 평가 마네킹 척도개발 및 표준화)

  • Kang, Eun-Ho;Choi, Jeong-Eun;Ham, Byung-Joo;Seok, Jeong-Ho;Lee, Kyoung-Uk;Kim, Won;Lee, Seung-Hwan;Lim, Hyun-Kook;Park, Young-Min;Yang, Jong-Chul;Ahn, Meekyung;Lee, Jae Seon;Chae, Jeong-Ho
    • Anxiety and mood
    • /
    • v.7 no.2
    • /
    • pp.113-118
    • /
    • 2011
  • Objectives : The Self-Assessment Manikin (SAM) developed by Bradley and Lang is a non-verbal pictorial assessment tool that measures human emotion and has been widely used. However, the pictures in SAM have not been easy for Korean subjects to understand or relate to. The authors developed a new manikin (Emotional Valence and Arousal Manikin, EVAS) modeling it after Korean faces while modifying and standardizing the SAM. Methods : Forty-one healthy subjects participated in this study. They were asked to rate the emotional valence and level of arousal using both the SAM and EVAS after being exposed to pictures of facial expressions for affective neurosciences-Korean version. The internal consistency of the EVAS and the correlation between EVAS and SAM were examined. Resuts : Internal consistencies of the valence using the EVAS were from 0.63 (surprise) to 0.82 (happiness) and those of the arousal were from 0.90 to 0.95. Correlation coefficients of the valence and arousal between the SAM and EVAS were from 0.61 (both surprise and disgust) to 0.84 (neutral) and from 0.82 (sadness) to 0.94 (fear), respectively. Conclusions : We developed a new manikin (the EVAS) for the Korean population modifying and standardizing the SAM. The EVAS demonstrated a good internal consistency and validity. As such, it can be used in the field of human emotion research.

A Study for the Conformity Assessment on the Natural Dye by HPLC (1) (HPLC를 활용한 천연염료 적합성 평가 연구 (1))

  • Lee, Eun Mi;Oh, Dong Ki;Yoon, Hong Jin;Kim, Jong Hoon
    • Fashion & Textile Research Journal
    • /
    • v.20 no.6
    • /
    • pp.752-758
    • /
    • 2018
  • Due to environmental pollution and human hazards of some synthetic dyes, the global fashion companies are increasingly interested in eco-friendly products, especially natural dye. For the globalization of natural dyeing industry based on this concern, however, there are some deficiencies of standardization, specification, and conformity assessment on natural dyestuffs as well as natural dyeing process. These obstacles should be not only studied but also defined for a commercial transaction. Accordingly, a study for conformity assessment on commercialized natural dyestuffs (red) was conducted by HPLC analysis in this study. As the results of HPLC analysis, alizarin and purpurin, representative index ingredients, were detected in most of the samples, but the index ingredient content in each sample was different. In addition, some samples showed the variety of peaks including the index ingredients and others. It was inferred that the representative index ingredients could be used on the traceability of natural sourced dyestuffs. These results are related to the index ingredient consistency, standardization, and reproducibility of natural dyed products including such as yarns, fabrics, garments, and so on. Therefore, the present study was demonstrated that in order to determine the conformity assessment system for the satisfaction of all stakeholders, the offering of information on the origin, manufacturing process, and index ingredient content of natural dyes should be prioritized.

Reliability and Validity of the Assessment Tool for Measuring Communication Skills in Nursing Simulation Education (시뮬레이션 교육에서 간호학생의 의사소통능력 측정도구의 신뢰도와 타당도 검증)

  • Yang, Hee Mo;Hwang, Seon Young
    • Korean Journal of Adult Nursing
    • /
    • v.28 no.1
    • /
    • pp.95-105
    • /
    • 2016
  • Purpose: This study was conducted to test whether Health Communication Assessment Tool (HCAT) developed by Campbell et al. in 2013 can be applied to assess Korean nursing students' communication skills in simulation education. Methods: The content validity of the Korean version of HCAT (K-HCAT) was evaluated through expert's assessment and a principal component analysis was conducted for testing construct validity. The convergent validity was tested by measuring relationship between the K-HCAT and those of existing communication assessment tool for standardized patient instruction. The evaluation was done by surveying 154 senior students from four different nursing colleges. Results: The K-HCAT was condensed into 15 items from the original 22 HCAT items. Four factors were extracted from the principal component analysis; factor loadings ranged from .50 to .83; cumulative explained variance was 62.65%. Four factors were entitled as 'relationship building', 'empowering', 'empathy/response', and 'education/feedback'. Cronbach's ${\alpha}$ for sub-dimensions ranged from .73 to .84. An evaluation of convergent validity showed that the scores of the K-HCAT were moderately correlated with those of an assessment tool for standardized patient instruction. Conclusion: The K-HCAT can be used as an effective tool for assessing nursing students' communication skills in various kinds of simulation educations Further research is needed to test the consistency of the K-HCAT.

Properties of the Measures to Assess Oxaliplatin-induced Peripheral Neuropathy: A Literature Review (옥살리플라틴 유도 말초신경독성 측정도구의 고찰)

  • Chu, Sang Hui;Lee, Yoon Ju;Lee, Young Joo;Cleeland, Charles S.
    • Journal of Korean Academy of Nursing
    • /
    • v.45 no.6
    • /
    • pp.783-801
    • /
    • 2015
  • Purpose: The purpose of this study is to provide a comprehensive overview of the various measures available for assessment of oxaliplatin-induced peripheral neuropathy (OXLIPN) and to evaluate the measurement properties of each assessment tool. Methods: A systematic review was conducted to identify existing measures for OXLIPN found in the databases of PubMed, Cochrane Library, Embase, RISS and KoreaMed. The quality of the 24 identified tools was evaluated based on their properties of measurement including content validity, internal consistency, criterion validity, construct validity, reproducibility, responsiveness, floor-ceiling effects and interpretability. Results: Ten (41.7%) of the 24 tools were identified as specific measures for assessing OXLIPN and the most popular type of measures were clinical grading systems by clinicians (58.3%) and only 29.2% of measures were identified as patient reported outcomes. The most frequently used tool was National Cancer Institute-Common Toxicity Criteria (NCI-CTC), but the validity of NCI-CTC has not been reported appropriately. Overall, the Neuropathic Pain Symptom Inventory (NPSI) received the best psychometric scores, and the Chemotherapy-induced Peripheral Neuropathy Assessment Tool (CIPNAT) and Functional Assessment of Cancer Therapy/Gynaecologic Oncology Group-neurotoxicity-12 (FACT/GOG-Ntx-12) followed NPSI. Conclusion: To select appropriate measure, evidences should be accumulated through the clinical use of tools. Therefore, practitioner and researchers are urged to report relevant statistics required for the validation of the currently used measures for assessment of OXLIPN.

A Study on the Application and Development of the Strategic Environmental Assessment(SEA) Model with Reference to Urban Comprehensive Plan (전략환경평가 모형의 개발과 적용에 관한 연구(I): 도시기본계획 평가를 중심으로)

  • Sung, Hyun-Chan;Kim, Kwi-Gon
    • Journal of Environmental Impact Assessment
    • /
    • v.6 no.1
    • /
    • pp.5-31
    • /
    • 1997
  • Strategic Environmental Assessment (SEA) is an assessment process integrated with policy and planning processes considering environmental impacts, not at the level of individual project, but at higher level of policy and plan. Presently, two systems for the SEA are being implemented in Korea. One is "the consultation of environmental investigation into a change in national land utilization", which is to be pre-discussed with the head of the related central administrative organizations, based on the respective individual code. The other is "the consultation of environmental investigation into an administrative plan and project", which is to be pre-discussed with the minister of the environments for the administrative plan. However, these two systems are not integrated procedures with the planning process. In other words, there is a separate post-environmental assessment after planning. Also concrete proposal is not yet prepared for the method and procedure of environmental investigation about details of the planning. Therefore, they do not playa role in taking precautions for environmentally sustainable development. In this study, for the effective environmental investigation at the level of planning, environmental assessment model to be integrated with urban comprehensive plans was developed. The results of this study can be summarized as follows: First, based on the analysis of systematic problems in environmental investigation of this country, the study compared and analysed the SEA method of foreign countries. derived the potential limitations in its applications to this country. Also, the new method in this study, is integrated into planning process by improving the limitations. Thus, we developed the SEA model for this country, which consists of seven steps ; Phase 1; establishing a work program, Phase 2; defining the scop. of the SEA, Phase 3; analyzing an existing environmental situation, Phase 4; examining the consistency with environmental policy, Phase 5; appraising and synthesizing the plan content, Phase 6; proposing recommendations, Phase 7; monitoring and feedback. Second, the two types of application program of the developed assessment model were proposed. One is to integrate this model into planning process, and the other is for the institute and ministry of environments to implement SEA after planning. An aim of this study was developing an environmental assessment model at the planning level which was not yet established in this country and was to apply the model to urban comprehensive plan for inspection. This research will make the effective operation of environmental investigation system possible at the planning level in this country and contribute to the environmental protection at the global level.

  • PDF