• 대한치과의사협회지
• /
• 제54권1호
• /
• pp.27-38
• /
• 2016

Objectives: This study evaluated the outcomes of continuing professional education in implant dentistry using Kirkpatrick's four-level evaluation model. Material and methods: The study was carried out through a questionnaire distributed to dentists who attended a continuing professional education in implant dentistry, 2008~2012. They were asked to fill out questionnaires at least 6 months after its completion. Results: Mean ages of total 23 dentist was $44.8^{\circ}{\ae}8.2$. Mean period after completion of education was $2.7^{\circ}{\ae}1.2$. Knowledge (level 2) (r=0.71, p<0.01) and behavior (level 3) (r=0.68, p<0.01) was positively correlated with topic and methods of education in reaction (level 1). Behavior was positively correlated with knowledge (r=0.79, p<0.01). Result (level 4) was positively correlated with knowledge (r=0.64, p<0.01) and behavior (r=0.86, p<0.01). Conclusion: Reaction affects on knowledge, behavior and result in order.

• 의학교육논단
• /
• 제23권1호
• /
• pp.56-67
• /
• 2021

Educational program evaluation can improve the quality of the curriculum, instructional methods, and resources and provide useful data for making educational decisions and policies. Developing and implementing a program evaluation system is essential in competency-based medical education. The purpose of this study was to explore and establish an educational program evaluation system adapting an integrated program evaluation model to promote competency-based medical education. First, an Educational Evaluation Committee was organized, consisting of faculty, staff members, and students. The committee established an integrated program evaluation model, combining Stufflebeam's Context, Input, Process, and Product (CIPP) model of a process-oriented approach and Kirkpatrick's four-level model of an outcome-oriented approach. Kirkpatrick's model was applied to the product evaluation of the CIPP model. The committee then developed evaluation criteria, indicators, and data collection methods according to the components of the CIPP model and the four levels (reaction, learning, behavior, and results) of Kirkpatrick's model, and collected and analyzed data. Finally, the committee reported the results of evaluation to a Medical Education Quality Improvement Committee, and the results were used to improve the curriculum and student selection. To enhance the quality of education, identifying educational deficiencies and developing various elements of education in a balanced way through educational evaluation will be needed. Furthermore, it will be necessary to listen to opinions of various stakeholders, work with all members involved in education, and communicate with decision-makers in the process of educational evaluation.

• 디지털융복합연구
• /
• 제19권12호
• /
• pp.51-62
• /
• 2021

본 연구는 전문경력인사 초빙활용지원사업의 성과 관리 및 평가 체계 강화를 위해 전문경력인사의 과업 수행에 대한 체계적·포괄적 평가가 가능한 평가 요소를 개발하였다. 이를 위해 경계이론과 커크패트릭 4수준 평가 모델에 기반을 두고 사업 평가와 관련한 기존 연구를 분석하여 성과 평가 프레임워크를 개발하였다. 이후 2차에 걸친 델파이 조사 방법을 활용하여 전문경력인사의 개인 측면과 활용 기관 측면에서 성과를 측정할 수 있는 평가 요소를 도출하고, 구체적인 측정 문항을 개발하여 타당화 작업을 진행하였다. 이러한 절차를 거쳐 최종적으로 적용성, 연계성, 명확성, 적합성, 확장성 등 5가지 평가 요소를 선정하였으며, 각 요소별 구체적인 평가 목적을 수립하여 실제 전문경력인사 초빙활용지원사업의 성과 관리를 위해 적용 가능하도록 하였다. 본 연구는 1994년부터 한국연구재단에서 운영하고 있는 전문경력인사 초빙활용지원사업의 성과 관리를 위해 정량적 정성평가가 가능한 성과 평가 체계와 요소를 제시하였다는 데 의의가 있다.

• 한국컴퓨터정보학회논문지
• /
• 제17권10호
• /
• pp.185-192
• /
• 2012

콜센터는 고객과의 접촉에 있어 비대면 채널에 의해 커뮤니케이션이 이루어지기 때문에 대면 접촉에 의한 것보다 더 많은 상담사의 능력을 요구한다. 이러한 상담사의 능력개발을 위해 콜센터들은 경력과 직무에 따라 다양한 교육훈련을 실시하고 있으며, 교육훈련의 성과로 상담품질이나 생산성 향상을 실현하고자 한다. 한편, 교육훈련에는 많은 시간과 예산이 투입되기 때문에 교육훈련의 평가를 통해 현업수행에 도움이 얼마나 되고 있는지 그 효과성이 파악되고 관리되어야 할 것이다. 지금까지의 교육훈련의 평가에 대한 연구들을 보면 만족도와 학습정도를 측정하거나 학습이 행동에 전이되는 정도를 측정하는 연구가 주류를 이루어 전체적인 평가모형에 관한 연구가 필요하다. 본 연구에서는 콜센터교육훈련의 효과성을 상담사들의 인식수준에서 Kirkpatrick의 4단계 평가모형을 반영하여 반응, 학습, 행위, 결과에 대한 각 단계별 평가기준 간에 어떠한 영향관계가 있는지 알아보고, 구조방정식 모델을 사용하여 전체적인 모형의 적합도를 살펴보았다, 또한 대안모형으로 반응요인과 행위요인의 직접적인 관계를 고려하여 연구모형과 대안모형의 구조모델 적합도를 비교 분석하였다.

• 보건행정학회지
• /
• 제26권4호
• /
• pp.373-381
• /
• 2016

Background: The purpose of this study is to evaluate the outcomes of clinical education program for nurses in regional public hospital, utilizing the Kirkpatrick's model. Methods: Kirkpatrik's 4-level model was applied to this study. Trainees were asked to fill out questionnaires in the middle and at the end of the program. Also administrators of excellent trainees were asked to fill out the questionnaires regarding nursing management performance after 1-2 months from the end of the training course. Results: All trainees had positive reactions to the clinical education program. Not only the results of individual level (satisfaction and achievement scores, academic achievement scores, practical application rate, and educational transition factors) but also the scores of organization level (nursing management performance scores) are improved. Conclusion: By showing a correlation between the effectiveness factors we need to verify the relationship between these factors in a future study. In addition, development of quantitative and qualitative performance indicators are needed. To establish a long-term education system, it is required to applying the excellent trainee's successful experiences.

• 경영과학
• /
• 제32권4호
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.

• 의학교육논단
• /
• 제25권2호
• /
• pp.126-131
• /
• 2023

The aim of this study was to systematically collect data for evaluating short- and long-term outcomes using Kirkpatrick's four-level evaluation model, Chonnam National Medical School has established plans for developing and managing a database of student and graduate cohorts. The Education Evaluation Committee, with assistance from the Medical Education Office, manages the development and maintenance of cohort data. Data collection began in the 2022 academic year with first- through fourth-year medical students and graduates of the year 2022. The collected data include sociodemographic characteristics, admission information, psychological test results, academic performance data, extracurricular activity data, scholarship records, national medical licensing exam results, and post-graduation career paths. The Education Evaluation Committee and the Medical Education Office analyze the annually updated student and graduate cohort data and report the results to the dean and relevant committees. These results are used for admissions processes, curriculum improvement, and the development of educational programs. Applicants interested in using the student and graduate cohort data to evaluate the curriculum or conduct academic research must undergo review by the Educational Evaluation Committee before being granted access to the data. It is expected that the collected data from student and graduate cohorts will provide a sound and scientific basis for evaluating short- and long-term achievements based on student, school, and other characteristics, thereby supporting medical education policies, innovation, and implementation.

• 한국치위생학회지
• /
• 제23권4호
• /
• pp.277-286
• /
• 2023

Objectives: In this study, we aimed to develop a systematic tool that can evaluate the effects of interprofessional education (IPE) by applying four core competencies (values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork) and an educational evaluation model to evaluate the learning, behavior of learners, and results step by step. Methods: Previous studies on IPE evaluation tools were analyzed, and an evaluation tool (draft) was developed by modifying questions suitable for evaluation according to the Kirkpatrick model's stages and core competencies. The evaluation tool was completed by conducting a Delphi survey twice with 4-6 experts. To analyze the Delphi survey, the content validity index (CVI) was calculated, and the reliability coefficient (Cronbach's alpha coefficient) was used to measure reliability. Results: 29 questions on pre-education, consisting of learning and behavior stages, and 54 questions on post-education, consisting of reaction, learning, behavior, and results stages, were developed. The CVI and Cronbach's alpha coefficient values were >0.8 and >0.6, respectively. Conclusions: The IPE evaluation tool developed in this study is expected to contribute to the evaluation of the educational level of IPE and the identification of points for improvement when applied to various educational settings.

• Journal of Welding and Joining
• /
• 제33권3호
• /
• pp.40-46
• /
• 2015

Welding is one of the most fundamental and necessary work in the industry that demand sophistication of skilled workers. This study is to introduce welding simulator as a training tool, to verify its effectiveness and to measure satisfaction of the trainees. A group of freshman students at a Korea Polytechnics College in their twenties with less experience of welding participated in the study. They were divided into two groups and took a traditional training course (comparison group) and a training course with welding simulator applied reality/haptic technology (experimental group) for same hours respectively. To evaluate training effect, a national certificate test and a survey based on Phillips' ROI (Return on Investment) methodology were conducted by the students and the college respectively. And satisfaction survey among the students based on Kirkpatrick's Four-Level Evaluation Model was also carried out. The results showed that all students in the experimental group passed the national certificate test and the ROI of the experimental group for five years were 110% higher than the comparison group. Furthermore, 25% more students in the experimental group replied "very satisfied" about the overall training course and 75% more students in the same group found that the simulation was very similar to the real welding.