• The Journal of the Korea Contents Association
• /
• v.21 no.3
• /
• pp.410-418
• /
• 2021

In a program that runs an engineering education certification, program outcomes refer to the knowledge, skills, and attitudes a student must have until graduation. In general, capstone design is used as a tool for evaluating program outcomes. This paper applies the intraclass correlation coefficient (ICC) to measure the raters' reliability in assessing program outcomes. Several raters evaluate program outcomes, and the result is used to obtain the raters' ICC. ICC measures the reliability of ratings or measurements for clusters - data that has been collected as groups or sorted into groups. If the ICC is close to 1, it means that the reliability among the raters is high. We evaluated the proposed method's usefulness through case analysis. As a method for assessing an evaluation tool's objectivity, multiple raters measure the same evaluation tool. As a result, we measured the ICC values for all POs, and analyzed the cause for the low measured POs. We applied this method to evaluate program outcomes of the Department of Computer Engineering in the past two years. As a result, we derived guidelines for improvement and program outcomes.