• 한국산학기술학회논문지
• /
• 제10권9호
• /
• pp.2424-2434
• /
• 2009

본 연구는 공기업을 중심으로 다면평가제도의 선행요인이 평가효용성에 미치는 영향과 평가효용성이 조직성과에 미치는 영향을 파악하고 평가제도의 사전교육정도에 따라 평가효용성에 미치는 영향에 차이가 있는지를 알아보고 궁극적으로 피평가자가 인식하는 평가효용성이 조직성과에 어떠한 영향을 미치는지 4가지 관점에서 연구하였다. 연구 결과 선행요인인 평가자 신뢰성, 최고경영자의 지지는 평가효용성에 유의한 영향을 미치고 있으나 평가준거는 유의적이지 못하였다. 사전교육의 정도에 따라 선행요인과 평가효용성 간에 조절효과가 있는지를 검증한 결과 평가자 신뢰성은 조절효과가 있는 것으로 판명되었다. 그러나 평가준거, 최고경영자의 지지는 조절효과가 없는 것으로 나타났다. 결과적으로 본 연구의 평가효용성이 선행요인의 영향을 받아서 조직의 성과 즉, 조직몰입도, 직무만족도, 자기계발기회, 조직활성화에 직접 또는 간접적으로 영향을 미치며, 사전교육이 부분적으로 조절효과가 있는 것으로 확인할 수 있었다.

• 한방안이비인후피부과학회지
• /
• 제26권2호
• /
• pp.10-18
• /
• 2013

Objectives : We performed a pilot study to investigate inter- and intra-rater reliability of pattern identification using nasal endoscopy for allergic rhinitis(AR). Methods : Eight experts of ophthalmology, otolaryngology and dermatology of Korean medicine evaluated 20 nasal endoscopy photograph cases of AR patients with pattern identification index using nasal endoscopy for AR including the nasal membrane color(pale / hyperemia), nasal membrane humidity(dryness / dampness), rhinorrhea(watery / yellow), and membrane edema (atrophic / edematous) on nasal endoscopy. Results : Intra-rater agreement(%) and Kappa coefficient was generally from 'moderate' to 'good'(% agreement: 73.13-90% / Kappa coefficient: 0.547-0.748). Inter-rater agreement(%) and Kappa coefficient was also from 'moderate' to 'good' (% agreement: 65-85% / Kappa: 0.475-0.778) except 'humidity(dryness / dampness)' item (% agreement: 55.98% / Kappa: 0.340). In findings of subgroup analysis according to affiliation of raters, Inter-rater agreement(%) and Kappa coefficient of raters in same affiliation was higher than inter-rater agreement(%) and Kappa coefficient of raters in different affiliation except 'dryness / dampness' item. Conclusions : It is necessary to improve objectivity and reproducibility of pattern identification using nasal endoscopy for allergic rhinitis(AR) through the development of detail-oriented criteria and enhanced training of clinicians with development of standard operating procedures(SOPs).

• 한국콘텐츠학회논문지
• /
• 제11권9호
• /
• pp.407-413
• /
• 2011

뇌졸중 환자의 실어증 유무를 정확히 판별하고 환자의 미세한 언어적 변화를 적절히 관찰하기 위해서는 일차적으로 신뢰로운 검사도구를 사용하여야 하고, 또한 검사자가 해당 도구의 사용에 있어 충분히 숙지하고 있어야 한다. 본 연구에서는 뇌졸중 환자의 실어증 및 말장애 유무의 진단에서 검사자의 전공영역에 따른 관찰자간 신뢰도를 살펴보고 훈련 전과 후의 차이를 비교해보고자 하였다. 이를 위해, 침상에 있는 뇌졸중 환자 46명을 대상으로 언어치료사, 신경과 전공의, 그리고 간호사 각각 4명씩 총 12명이 동시에 실어증 및 말장애 유무를 평가하였다. 연구 결과, 서로 다른 전공 영역의 전문가들 간에 말명료도 과제와/아/모음연장발성 과제의 '음질' 영역에서만 'acceptable'로 나타났고, 나머지 하부검사 영역에서는 'good-excellent'로 나타났다. 관찰자간 신뢰도가 'acceptable'로 나타난 과제들에 대해 3주간의 비디오 훈련 전후의 점수 차이를 비교하였다. 그 결과, 훈련 후 말명료도 과제에서 검사자들 간의 평정 점수의 차이는 유의하게 줄어들었으며, '음질' 평정의 정확성도 유의하게 증가하였다. 임상 경험 정도와 각 하부검사에 대한 평정 정확성 간의 상관관계를 알아본 결과, 언어치료사들은 임상 경험 정도가 증가할수록 그림설명하기 과제와 말명료도 과제에서, 그리고 의사 및 간호사들은 그림설명하기 과제에서 판정 정확도가 높아지는 것으로 나타났다. 결론적으로, 이 연구 결과는 뇌졸중 환자의 의사소통장애 진단에 있어 신경언어장애 환자 중, 특히 말장애환자에 대한 꾸준한 경험과 훈련이 반드시 필요하며, 훈련을 통해 평정 신뢰도를 확보할 수 있음을 시사한다.

• 대한기계학회논문집B
• /
• 제41권2호
• /
• pp.125-133
• /
• 2017

본 연구에서는 환자의 손가락 경직을 모사하는 손가락 시뮬레이터를 통해 손가락 경직도 검진에서의 검진자간 신뢰도에 대해 연구하였다. 시뮬레이터를 제어하기 위해 경직에서 나타나는 토크를 간단하게 모델링 하고 손가락 경직 환자로부터 간단한 측정 모듈을 이용해 각각의 Modified Ashworth Scale(MAS) 등급에 맞는 파라미터를 얻어냈다. 또한 중수지 관절에 모터가 위치한 손가락 형태의 로봇을 설계하여 경직 토크 모델을 따르는 시뮬레이터를 개발하였다. 이 시뮬레이터를 통해 일곱 명의 재활의학과 전문의들의 검진 결과를 평가해본 결과 중수지 관절에서 0.619, 근위지간 관절에서 0.514의 Cohen's Kappa 값을 보였다. 검진자간의 Fleiss' Kappa 값은 중수지 관절에서 0.513, 근위지간 관절에서 0.486으로 나타났다. 또한 검진자들은 각각의 주관적인 MAS 검진 기준을 가진다는 것을 확인하였다. 결과적으로 같은 환자에 대한 MAS 검진 신뢰도가 높지 않기 때문에 개발된 로봇 시뮬레이터는 검진자의 신뢰도를 높일 수 있는 교육용 도구로서 활용 가능함을 확인하였다.

• The Journal of Korean Physical Therapy
• /
• 제22권5호
• /
• pp.83-93
• /
• 2010

Purpose: The purpose of this study was to determine the reliability and validity of the Functional Movement Screen (FMS) for assessing Korean athletes. Methods: A total of 48 patients (37 males and 11 females) participated in this study. Data were acquired after translation of the FMS from English to Korean and cross-cultural adaptation of the this questionnaire. To determine inter-rater reliability, the relationship between the FMS scores obtained by two raters was evaluated using the Kappa coefficient, which was in total agreement with the Intra-Correlation Coefficient (ICC3,1). Concurrent validity was examined by correlating the FMS scores with the Oswestry disability index (ODI) scores and Visual analogue scale (VAS) scores. Results: The raters demonstrated excellent agreement on 7 (above 90%) of the 17 test (72.9 to 97.9%) components. Substantial agreement was seen in 11 of the 17 tests. Two components of the In-line lunge and rotatory stability tests demonstrated moderate agreement. It showed good inter-rater reliability: the Kappa coefficients ranged from 0.42 to 0.97. ICC3,1=0.42~0.99 and 0.93 (total FMS score). Cronbach's alpha for FMS was 0.80. It was not correlated with ODI or VAS. Conclusion: The Korean version of the FMS is a reliable instrument for measuring movement patterns of Korean athletes and for making decisions related to interventions for performance enhancement.

• Clinics in Shoulder and Elbow
• /
• 제25권4호
• /
• pp.321-327
• /
• 2022

Background: Scapular dyskinesis is considered a risk factor for the shoulder pain that may warrant screening for prevention. Clinicians of all experience screen scapular dyskinesis using the scapular dyskinesis test yes-no classification (Y-N), yet its reliability in asymptomatic individuals is unknown. We aimed to establish Y-N's intra- and inter-reliability between students and expert physical therapists. Methods: We utilized a cross-sectional design using consecutive asymptomatic subjects. Six students and two experts rated 100 subjects using the Y-N. Cohen's kappa (κ) and Krippendorff's alpha (K-α) were calculated to determine intra- and inter-rater reliability. Results: Intra- and inter-rater values for experts were κ=0.92 (95% confidence interval [CI], 0.91-0.93) and 0.85 (95% CI, 0.84-0.87) respectively; students were κ=0.77 (95% CI, 0.75-0.78) and K-α=0.63 (95% CI, 0.58-0.67). Conclusions: The Y-N is reliable in detecting scapular dyskinesis in asymptomatic individuals regardless of experience.

• The Journal of Advanced Prosthodontics
• /
• 제14권4호
• /
• pp.203-211
• /
• 2022

PURPOSE. The aim of this study was to introduce rating criteria to evaluate student performance in a newly developed, digital wax-up preclinical program for computer-aided design (CAD) of full-coverage crowns and preliminarily investigate the reliability and internal consistency of the rating system. MATERIALS AND METHODS. This study, conducted in 2017, enrolled 47 fifth-year dental students of Okayama University Dental School. Digital wax-up training included a fundamental practice using computer graphics (CG), multipurpose CAD software programs, and an advanced practice to execute a digital wax-up of the right mandibular second molar (#47). Each student's digital wax-up work (stereolithography data) was evaluated by two instructors using seven qualitative criteria. The total qualitative score (0-90) of the criteria was calculated. The total volumetric discrepancy between each student's digital wax-up work and a reference prepared by an instructor was automatically measured by the CAD software. The inter-rater reliability of each criterion was analyzed using a weighted kappa index. The relationship between the total volume discrepancy and the total qualitative score was analyzed using Spearman's correlation. RESULTS. The weighted kappa values for the seven qualitative criteria ranged from 0.62 - 0.93. The total qualitative score and the total volumetric discrepancy were negatively correlated (ρ = -0.27, P = .09, respectively); however, this was not statistically significant. CONCLUSION. The established qualitative criteria to evaluate students' work showed sufficiently high inter-rater reliability; however, the digitally measured volumetric discrepancy could not sufficiently predict the total qualitative score.

• 한국간호교육학회지
• /
• 제22권4호
• /
• pp.549-558
• /
• 2016

Purpose: The purpose of this study was to develop and validate a Nursing Competency Scale in Simulation (NCSS) for nursing students. Methods: A preliminary version of the NCSS of 14 items was derived from the literature. A panel of seven experts reviewed the preliminary version for content validation and developed 15 items scale. A convenient sample of 195 nursing students completed the survey and two evaluators measured the performances of nine teams in a simulation scenario with NCSS. The data were analyzed using exploratory factor analysis, descriptive statistics, independent t-test and Cronbach's alpha. Intraclass correlation coefficient (ICC) was used in order to estimate the degree of inter-rater reliability. Results: An exploratory factor analysis demonstrated that two factor structures of the NCSS explained 51.1% of the total variance. Two factors were named psychosocial skills and cognitive and psychomotor skills. The mean scores of NCSS between third and 4th grade were significantly different providing support for its known-group validity. Cronbach's alpha was .90 and ranged from .79 to .88. The overall ICC for inter-rater agreement was 0.89 (95% CI 0.03 to 0.98). Conclusion: This scale shows preliminary evidence for validity and reliability. It could be a useful instrument for measuring learning outcome in simulation for nursing students' clinical competency.

• 대한물리의학회지
• /
• 제8권1호
• /
• pp.137-145
• /
• 2013

PURPOSE: The purpose of this study is to examine assessment of test-retest reliability for 13 items of pediatric balance scales, then the intrarater reliability among the raters. METHODS: Study participants included 6 children with spastic cerebral palsy who could walk. Raters were consist of seventy-four, 2nd year physical therapy major students. The children's ability to achieve physical balance was wideotaped for PBS items. The raters watched the tapes and evlauated each child twice. Test-retest reliability was analyzed using the Spearman correlation, and interrater reliability was analyzed using the Kendall's coefficient of concordance for ranks. RESULTS: The total PBS scores averaged 49.22 and 50.06 for first and second tests. Test-retest reliability of PBS individual items were between low and high. The low itmes were 3 items, and high were 4 itmes. Interrater reliability were low agreement. CONCLUSION: Based on the study, the pediatric balance scale is an effective evaluating tool for measuring functional balance of school age child. However, the agreement of students or beginner's interpretation is necessary in conjunction to this study results. On top of that, it is important to provide detailed explanation and repetitive training, which improves the analysis reliability. Also, each category subscores level for the pediatric balance scale must be examined carefully in conjunction to performing test, only after improving match quality via preceding practice.

• The Journal of Korean Physical Therapy
• /
• 제27권4호
• /
• pp.175-182
• /
• 2015

Purpose: The purposes of this study were to develop the Korean version of the trunk impairment scale (K-TIS) and to examine reliability and responsiveness of the K-TIS in patients with stroke. Methods Subjects of the study were 51 stroke patients (mean age: 57.78 years) recruited from two stroke clinics. For the interrater and test-retest reliability, two raters measured the K-TIS two times using video clips with an interval of 2 weeks. For the responsiveness, intensive physical therapy training was provided to all participants 2 times a day for one month or three months depending on the onset of the stroke and the admission rules of the two clinics. Inter-rater reliability and test-retest reliability of the K-TIS three subscales (static sitting balance, dynamic sitting balance, and coordination) scores and total scores were examined using intra-correlation coefficient ($ICC_{3,1}$) and Pearson's correlation coefficient (r). To examine responsiveness, the minimally important difference (MID) was calculated with effect size. Results: Inter-rater reliability of the K-TIS subscales and total scores were all high (ICC3,1=0.920-0.983 and r=0.924-0.984). For the test-retest reliability, $ICC_{3,1}$=0.805-0.901 and r=0.806-0.903, and the MID for acute and post-acute as well as chronic stroke patients remained in the mean change range. Conclusion: It is suggested that the K-TIS might be used for clinical and research purposes as a standardized tool for stroke patients. In addition, it can also be useful in establishment of treatment goal(s) and planning treatment program(s) for patients with stroke.