• Journal of Preventive Medicine and Public Health
• /
• 제48권5호
• /
• pp.239-248
• /
• 2015

Objectives: The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events. Methods: We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later. Results: In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89). Conclusions: In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.

• 한국전문물리치료학회지
• /
• 제17권2호
• /
• pp.10-16
• /
• 2010

This study was undertaken to develop the scapulometer, a measuring tool for scapular winging, and to determine the reliability of scapulometer. Thirty subjects with scapular winging greater than 2 cm participated in this study. Two examiners measured scapular winging using the scapulometer. Standardized measurement position of each subject was as follows: neutral position of shoulder and wrist, $90^{\circ}$ flexion of elbow, and cuff weight of 5% and 10% of body weight (BW) to neutral wrist. The interrater and intrarater reliability were calculated with an intraclass correlation coefficient (ICC) with 95% confidence interval (CI) and standard error of measurement (SEM) at 5% BW and 10% BW. The level of scapular winging was tested between 5% BW and 10% BW with a paired t-test. The interrater reliability was shown to be high at 5% BW (ICC(2,k)=.955, 95% CI=.900~.979, SEM=.07) and at 10% BW (ICC(2,k)=.968, 95% CI=.930~.985, SEM=.06). The intrarater reliability was shown to be high at 5% BW (ICC(2,k)=.921, 95% CI=.842~.961, SEM=.09) and at 10% BW (ICC(2,k)=.906, 95% CI=.813~.954, SEM=.10). The level of scapular winging was significantly different between 5% BW and 10% BW. Conclusively, it is demonstrated that the scapulometer is an objective and qualitative measurement tool for scapular winging.

• The Journal of Korean Physical Therapy
• /
• 제25권4호
• /
• pp.174-179
• /
• 2013

Purpose: Visual gait analysis plays a pivotal role in determining the important gait problem of patients. A few studies have been published and have received little attention regarding visual gait analysis on patients with orthopedic problems. The purposes of this study were to investigate the difference of reliability levels according to experience of clinical physical therapists. Methods: Thirty-five clinical physical therapists, 5 high experienced, 15 experienced, and 15 inexperienced, were recruited and individually evaluated these videotaped gait patterns of the participants, and filled up the structured gait analysis form. The gait of nine participants was videotaped. Reliability levels were calculated by the Intraclass Correlation Coefficients (ICC). Results: The inter-rater reliability of high experienced group (ICC=0.56; 95% CI: 0.50-0.62) was comparable to that of the experienced raters (ICC=0.48; 95% CI: 0.43-0.53) and inexperienced group (ICC=0.42; 95% CI: 0.38-0.46). High experienced group reached a higher inter-rater reliability level. The average intra-rater reliability of the high experienced group was 0.70 (ICCs ranging from 0.54 to 0.82). The experienced group reached an average intra-rater reliability of 0.61 (ICCs ranging from 0.47 to 0.81). The inexperienced group attained average ICC values of 0.53 (ICCs ranging from 0.30 to 0.74). Conclusion: Use of a structured gait analysis form as described in this study was found to be moderately reliable. Clinical experience appears to increase the reliability of visual gait analysis.

• 시스템엔지니어링학술지
• /
• 제4권1호
• /
• pp.35-43
• /
• 2008

Generally, in analysis of reliability of Design and Development Phase, reliability of electrical components is analyzed based on standards such as MIL-HDBK-217F, Bellcore Issue 4,5,6 by analyzing stress of architectural side (Power, Voltage, Current and quality level of components) of weapon system and stress of operational side (operational environment, operational temperature, Operational Profile). But the reliability of mechanical components is analyzed based on the data book of failure history of mechanical parts called NPRD-95(Nonelectronic Parts Reliability Data-95) without any analysis of above stress. However, even if it's the same mechanical parts, it might have different failure rate(fatigue, wear, corrosion) during operation depending on how weary(stress : pressure, vibration, temperature during operation) the parts are. Therefore, analyzing reliability using just data book can cause big difference in reliability instead of analyzing based upon stressfulness that parts might have, operational concept, and other various factors. Thus, This paper will guide the way of predicting reliability by organizing ways of predicting reliability for system organization and adopt ing NSWC-98/LE1(Naval Surface Warfare Center-98/LE1) for mechanical components.

• Korean Journal of Radiology
• /
• 제20권6호
• /
• pp.880-893
• /
• 2019

Objective: To assess the technical performance of two-dimensional shear wave elastography (2D-SWE) for measuring liver stiffness. Materials and Methods: The Ovid-MEDLINE and EMBASE databases were searched for studies reporting the technical performance of 2D-SWE, including concerns with technical failures, unreliable measurements, interobserver reliability, and/or intraobserver reliability, published until June 30, 2018. The pooled proportion of technical failure and unreliable measurements was calculated using meta-analytic pooling via the random-effects model and inverse variance method for calculating weights. Subgroup analyses were performed to explore potential causes of heterogeneity. The pooled intraclass correlation coefficients (ICCs) for interobserver and intraobserver reliability were calculated using the Hedges-Olkin method with Fisher's Z transformation of the correlation coefficient. Results: The search yielded 34 articles. From 20 2D-SWE studies including 6196 patients, the pooled proportion of technical failure was 2.3% (95% confidence interval [CI], 1.3-3.9%). The pooled proportion of unreliable measurements from 20 studies including 6961 patients was 7.5% (95% CI, 4.7-11.7%). In the subgroup analyses, studies conducting more than three measurements showed fewer unreliable measurements than did those with three measurements or less, but no intergroup difference was found in technical failure. The pooled ICCs for interobserver reliability (from 10 studies including 517 patients) and intraobserver reliability (from 7 studies including 679 patients) were 0.87 (95% CI, 0.82-0.90) and 0.93 (95% CI, 0.89-0.95), respectively, suggesting good to excellent reliability. Conclusion: 2D-SWE shows good technical performance for assessing liver stiffness, with high technical success and reliability. Future studies should establish the quality criteria and optimal number of measurements.

• Physical Therapy Rehabilitation Science
• /
• 제9권2호
• /
• pp.120-130
• /
• 2020

Objective: This study was conducted to identify the clinical usefulness, validity, and reliability of the Spirokit, a device that combines the pulmonary function test (PFT) and respiratory muscle strength (RMS) test. Design: Cross-sectional study. Methods: Forty young adults (male: 23, female: 17) participated in a PFT and a RMS test. The concurrent validity for pulmonary function was assessed by comparing data obtained from MicroQuark and the Spirokit and the agreements between the MicroRPM and the Spirokit for RMS were compared. The test-retest reliability of the Spirokit was determined by comparing data obtained from the first and second sessions. The test and retest were performed at the same time after one day for the PFT and RMS test. Validity was estimated using intraclass correlation coefficients (ICCs), and by calculating 95% limits of agreement (LoA). To estimate interrater reliability, ICCs were calculated. Results: The Spirokit showed a high agreement intra class coefficient (ICC [2, 1]): 0.978-0.999, 95% limits of agreements (95% LOA): -0.798 to 0.847 with MicroQuark. It also showed a high level of concordance ICC (2, 1): 0.992 to 0.993, 95% LOA: -9.313 to 11.169 with MicroRPM. The test-retest reliability of the Spirokit was analyzed using ICC (2, 1), and showed a high level of reliability (ICC [2,1]=0.960 to 0.998). Standard error of measurement % (SEM%) was 0.12% to 3.39%, and minimum detectable change% (MDC%) was 0.02% to 3.79%, indicating high level of reliability. Conclusions: The Spirokit is a device with high validity and reliability that can be used to simultaneously measure PFT and RMS tests.

• 환경위생공학
• /
• 제22권4호
• /
• pp.1-9
• /
• 2007

A wastewater treatment plant, operating process is physico-chemical/biological activated sludge attached sand filtration, was selected to evaluate effluent quality and pollutants removal efficiencies consideration for deriving the technology-based effluent limitation for petroleum refining industry discharge. The results of influent and effluent analysis were as follows: Average effluent quality were 0.076mg/l of copper, 0.084mg/l of lead, 0.036mg/l of zinc, 0.005mg/l of nickel and 0.004mg/l of cadmium, and the range of coefficient of reliability from 0.007 of copper to 1.0 of lead. Also, 95% of reliability, 0.112, 0.15, 0.063, 0.015 and 0.009mg/l, respectively, were remarkably lower than their effluent limitations. And to reach 95% reliability of effluent limitation at cleanness area, designed effluent quality of copper, lead, zinc and cadmium should be 0.268, 0.099, 0.526 and 0.008mg/l, respectively.

• 디지털콘텐츠학회 논문지
• /
• 제19권7호
• /
• pp.1405-1412
• /
• 2018

한국어 의지평가 설문지를 개발하고 그 타당도가 연구되었으나, 다리 절단장애인을 대상으로 의지평가 설문지를 적용하여 신뢰도를 평가한 연구는 실시되지 않았다. 본 연구에서는 다리 절단장애인을 대상으로 개발된 한국어-PEQ를 적용하여 그에 대한 신뢰도를 평가하였다. 그 결과, 한국어-PEQ의 모든 항목에서 천장효과와 바닥효과가 15%이하로 나타나 응답의 난이도가 적절한 것으로 나타났다. 한국어-PEQ 영역별 신뢰도(inter-rater reliability)의 급간내 상관계수(intra-class coefficient, ICC)는 .719(95%CI .600~.811)~.920(95%CI .890~.945)로 나타났고, 영역간 신뢰도는 .958로 높게 나타났다. 문항 내적 일치도 Cronbach's ${\alpha}$값은 .910으로 높게 나타나 한국어-PEQ는 높은 신뢰도가 있음을 알 수 있었다.

• Clinics in Shoulder and Elbow
• /
• 제25권4호
• /
• pp.321-327
• /
• 2022

Background: Scapular dyskinesis is considered a risk factor for the shoulder pain that may warrant screening for prevention. Clinicians of all experience screen scapular dyskinesis using the scapular dyskinesis test yes-no classification (Y-N), yet its reliability in asymptomatic individuals is unknown. We aimed to establish Y-N's intra- and inter-reliability between students and expert physical therapists. Methods: We utilized a cross-sectional design using consecutive asymptomatic subjects. Six students and two experts rated 100 subjects using the Y-N. Cohen's kappa (κ) and Krippendorff's alpha (K-α) were calculated to determine intra- and inter-rater reliability. Results: Intra- and inter-rater values for experts were κ=0.92 (95% confidence interval [CI], 0.91-0.93) and 0.85 (95% CI, 0.84-0.87) respectively; students were κ=0.77 (95% CI, 0.75-0.78) and K-α=0.63 (95% CI, 0.58-0.67). Conclusions: The Y-N is reliable in detecting scapular dyskinesis in asymptomatic individuals regardless of experience.

• The Journal of Korean Physical Therapy
• /
• 제28권2호
• /
• pp.119-123
• /
• 2016

Purpose: The purpose of this study was to determine the intra- and inter-rater reliability of muscle thickness (MT) measurements of the abductor hallucis (AbdH) in subjects with hallux valgus (HV), using ultrasonography performed at different inward pressures of approximately 0.5 kg, 1.0 kg, 1.5 kg, and 2.0 kg, with no pressure control. Methods: Thirty-two subjects with HV were recruited. The thicknesses of both sides of the AbdH were measured randomly by two different examiners for assessment of the intra- and inter-rater reliability. The measurement values were analyzed using the intra-class correlation coefficient (ICC) with a 95% confidence interval (CI). ICC (2,1) was used to determine the inter-rater reliability of MT measurements of the AbdH, while ICC (3,1) was used to assess the intra-rater reliability. Results: The results showed higher ICC values for intra-rater reliability compared to inter-rater reliability, and the value for inter-rater reliability with no pressure control (ICC=0.74 [95%CI=0.53-0.87]) was smaller compared to pressures of 0.5 kg, 1.0 kg, 1.5 kg, and 2.0 kg. Other inward pressures for intra- and inter-rater reliability also showed excellent values (ICC=0.86-0.96). Conclusion: The findings showed that maintaining consistent inward pressure is essential for reliable results in measurement of the MT of the AbdH by different examiners in a clinical setting.