• The Journal of the Korean dental association
• /
• v.58 no.5
• /
• pp.284-291
• /
• 2020

This study analyzed the change in the rater reliability based on the student's practice evaluation process conducted at Yonsei University College of Dentistry. Through this, we suggest the significance of the rater calibration training in the student's practical evaluation of dental college. Nine professors from the department of Conservative Dentistry, Yonsei University College of Dentistry, analyzed the results of class II restoration cases twice in 2017 and once in 2018. Intra Class Correlation (ICC) which is a statistic used to determine the consistency of raters with three or more scores, was also calculated. ICC values increased as raters participated in rater calibration meetings and grading experiences. This shows that the rater reliability is related to the grading experience and feedback from calibration meeting. Based on the results of previous studies that grading experiences and rater calibration training can cause a meaningful change in rater behavior, we propose to conduct rater calibration training to ensure the evaluator reliability.

• Journal of Oriental Neuropsychiatry
• /
• v.31 no.1
• /
• pp.1-12
• /
• 2020

Objectives: The purpose of this study was to evaluate the reliability and validity of the instrument on pattern identification for insomnia (PIT-Insomnia) and verify the correlation between PIT-Insomnia and psychological tests. Methods: Two evaluators examined the pattern identification of the participants who met insomnia disorder diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorder, Fifth Edition (DSM-5) and took the Insomnia Severity Index (ISI) score over 15 once manually and twice using the PIT-Insomnia to measure the inter-rater and test-retest reliability. We also conducted the following surveys: the Pittsburgh Sleep Quality Index (PSQI), the Korean version of Beck's depression inventory (K-BDI), the Korean version of the State-Trait Anxiety Inventory (STAI-K), the Korean Symptom checklist-95 (KSCL-95), and the EuroQol-5 dimension (EQ-5D), to measure concurrent validity and correlation between the PTI-Insomnia and psychological tests. Results: 1. The test-retest reliability analysis of the pattern identification results showed moderate agreement, and test-retest reliability analysis of each pattern identification score showed agreements from poor to moderate. 2. The inter-rater reliability analysis of the pattern identification results via manual showed slight agreement, when analysis was performed with calibration, the inter-rater reliability analysis of the pattern identification results via manual showed fair agreement. 3. The concordance analysis between results via manual and the PIT-Insomnia showed poor agreement, when the analysis was performed with calibration, concordance analysis showed fair agreement. 4. The concordance analysis between the PIT-Insomnia and the PSQI showed positive linear correlation. 5. The concordance analysis between the PIT-Insomnia and the PSQI, K-BDI, STAI-K, KSCL-95, and EQ-5D showed that non-interaction between the heart and kidney have positive linear correlation with the K-BDI, anxiety item of KSCL-95, dual deficiency of the heart-spleen have positive linear correlation with somatization item of KSCL-95, paranoia item of KSCL-95, heart deficiency with timidity have positive linear correlation with stress vulnerability item of KSCL-95, parania item of KSCL-95, phlegm-fire harassing the heart have positive linear correlation with K-BDI, paranoia item of KSCL-95, depressed liver qi transforming into fire have positive linear correlation with the anxiety item of KSCL-95, parania item of KSCL-95, all pattern identification have negative linear correlation with EQ-5D. Conclusions: The PIT-Insomnia has moderate agreement of reliability and reflects the severity of insomnia since it has some concurrent validity with the PSQI. There are some correlations between the PTI-Insomnia with specific psychological tests, so we could suggest it can be used appropriately in the clinical situation.