• The Journal of the Korean dental association
• /
• v.54 no.1
• /
• pp.27-38
• /
• 2016

Objectives: This study evaluated the outcomes of continuing professional education in implant dentistry using Kirkpatrick's four-level evaluation model. Material and methods: The study was carried out through a questionnaire distributed to dentists who attended a continuing professional education in implant dentistry, 2008~2012. They were asked to fill out questionnaires at least 6 months after its completion. Results: Mean ages of total 23 dentist was $44.8^{\circ}{\ae}8.2$. Mean period after completion of education was $2.7^{\circ}{\ae}1.2$. Knowledge (level 2) (r=0.71, p<0.01) and behavior (level 3) (r=0.68, p<0.01) was positively correlated with topic and methods of education in reaction (level 1). Behavior was positively correlated with knowledge (r=0.79, p<0.01). Result (level 4) was positively correlated with knowledge (r=0.64, p<0.01) and behavior (r=0.86, p<0.01). Conclusion: Reaction affects on knowledge, behavior and result in order.

• Korean Medical Education Review
• /
• v.23 no.1
• /
• pp.56-67
• /
• 2021

Educational program evaluation can improve the quality of the curriculum, instructional methods, and resources and provide useful data for making educational decisions and policies. Developing and implementing a program evaluation system is essential in competency-based medical education. The purpose of this study was to explore and establish an educational program evaluation system adapting an integrated program evaluation model to promote competency-based medical education. First, an Educational Evaluation Committee was organized, consisting of faculty, staff members, and students. The committee established an integrated program evaluation model, combining Stufflebeam's Context, Input, Process, and Product (CIPP) model of a process-oriented approach and Kirkpatrick's four-level model of an outcome-oriented approach. Kirkpatrick's model was applied to the product evaluation of the CIPP model. The committee then developed evaluation criteria, indicators, and data collection methods according to the components of the CIPP model and the four levels (reaction, learning, behavior, and results) of Kirkpatrick's model, and collected and analyzed data. Finally, the committee reported the results of evaluation to a Medical Education Quality Improvement Committee, and the results were used to improve the curriculum and student selection. To enhance the quality of education, identifying educational deficiencies and developing various elements of education in a balanced way through educational evaluation will be needed. Furthermore, it will be necessary to listen to opinions of various stakeholders, work with all members involved in education, and communicate with decision-makers in the process of educational evaluation.

• Journal of Digital Convergence
• /
• v.19 no.12
• /
• pp.51-62
• /
• 2021

This study developed the factors capable of systematic/comprehensive evaluation of the task performance in order to strengthen the performance management of the professional career personnel invitation program (PCPIP). To this end, a performance evaluation framework was developed by analyzing existing project evaluation studies based on boundary theory and Kirkpatrick's four-level evaluation model. Afterwords, through two Delphi surveys, evaluation factors that can measure performance in terms of individual and invitation institutions of PCP were derived and validated. With this procedure, five evaluation factors were finally selected: adaptability, connectivity, clarity, compatibility, and expandability. This study has implications suggesting a performance evaluation factors capable of hybrid quantitative/qualitative evaluation for the performance management of PCPIP operated by National Research Foundation of Korea Research since 1994.

• Journal of the Korea Society of Computer and Information
• /
• v.17 no.10
• /
• pp.185-192
• /
• 2012

Call Center requires an ability of agents a lot more than face-to-face contact due to being achieved communication by non face-to face channel for contact with customers. In order to improve the ability of agents, Call Center carries out various educational training according to their work experience and function and with the accomplishment of educational training, Call Center is going to fulfill to develop its quality of counseling and productivity. On the other hand, due to investment of a lot of time and budget to educational training, it is needed to grasp and manage about its effectiveness that how helpful the training is for performance of work-site operations through evaluation of educational training. Having Seen researches about evaluation of educational training until these days, most researches have mainstream to measure satisfaction and a level of learning or degree that how the learning transfers to actions. It is found that a research about an entire evaluation model should be required. This study aims to investigate effectiveness of Call Center educational training from the level of recognition by reflecting Kirkpatrick's the four levels of learning evaluation. By the four levels, reaction, learning, behavior and results, the study found out a connection with standards of evaluation about each levels. In addition, by using structural equation modeling, it was examined goodness of fit about the entire model. Furthermore, by an alternative model, considering a direct relation between a factor of reaction and behavior, it was compared and examined goodness of fit of overall model of the study model and the alternative one.

• Health Policy and Management
• /
• v.26 no.4
• /
• pp.373-381
• /
• 2016

Background: The purpose of this study is to evaluate the outcomes of clinical education program for nurses in regional public hospital, utilizing the Kirkpatrick's model. Methods: Kirkpatrik's 4-level model was applied to this study. Trainees were asked to fill out questionnaires in the middle and at the end of the program. Also administrators of excellent trainees were asked to fill out the questionnaires regarding nursing management performance after 1-2 months from the end of the training course. Results: All trainees had positive reactions to the clinical education program. Not only the results of individual level (satisfaction and achievement scores, academic achievement scores, practical application rate, and educational transition factors) but also the scores of organization level (nursing management performance scores) are improved. Conclusion: By showing a correlation between the effectiveness factors we need to verify the relationship between these factors in a future study. In addition, development of quantitative and qualitative performance indicators are needed. To establish a long-term education system, it is required to applying the excellent trainee's successful experiences.

• Korean Management Science Review
• /
• v.32 no.4
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.

• Korean Medical Education Review
• /
• v.25 no.2
• /
• pp.126-131
• /
• 2023

The aim of this study was to systematically collect data for evaluating short- and long-term outcomes using Kirkpatrick's four-level evaluation model, Chonnam National Medical School has established plans for developing and managing a database of student and graduate cohorts. The Education Evaluation Committee, with assistance from the Medical Education Office, manages the development and maintenance of cohort data. Data collection began in the 2022 academic year with first- through fourth-year medical students and graduates of the year 2022. The collected data include sociodemographic characteristics, admission information, psychological test results, academic performance data, extracurricular activity data, scholarship records, national medical licensing exam results, and post-graduation career paths. The Education Evaluation Committee and the Medical Education Office analyze the annually updated student and graduate cohort data and report the results to the dean and relevant committees. These results are used for admissions processes, curriculum improvement, and the development of educational programs. Applicants interested in using the student and graduate cohort data to evaluate the curriculum or conduct academic research must undergo review by the Educational Evaluation Committee before being granted access to the data. It is expected that the collected data from student and graduate cohorts will provide a sound and scientific basis for evaluating short- and long-term achievements based on student, school, and other characteristics, thereby supporting medical education policies, innovation, and implementation.

• Journal of Korean society of Dental Hygiene
• /
• v.23 no.4
• /
• pp.277-286
• /
• 2023

Objectives: In this study, we aimed to develop a systematic tool that can evaluate the effects of interprofessional education (IPE) by applying four core competencies (values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork) and an educational evaluation model to evaluate the learning, behavior of learners, and results step by step. Methods: Previous studies on IPE evaluation tools were analyzed, and an evaluation tool (draft) was developed by modifying questions suitable for evaluation according to the Kirkpatrick model's stages and core competencies. The evaluation tool was completed by conducting a Delphi survey twice with 4-6 experts. To analyze the Delphi survey, the content validity index (CVI) was calculated, and the reliability coefficient (Cronbach's alpha coefficient) was used to measure reliability. Results: 29 questions on pre-education, consisting of learning and behavior stages, and 54 questions on post-education, consisting of reaction, learning, behavior, and results stages, were developed. The CVI and Cronbach's alpha coefficient values were >0.8 and >0.6, respectively. Conclusions: The IPE evaluation tool developed in this study is expected to contribute to the evaluation of the educational level of IPE and the identification of points for improvement when applied to various educational settings.

• Journal of Welding and Joining
• /
• v.33 no.3
• /
• pp.40-46
• /
• 2015

Welding is one of the most fundamental and necessary work in the industry that demand sophistication of skilled workers. This study is to introduce welding simulator as a training tool, to verify its effectiveness and to measure satisfaction of the trainees. A group of freshman students at a Korea Polytechnics College in their twenties with less experience of welding participated in the study. They were divided into two groups and took a traditional training course (comparison group) and a training course with welding simulator applied reality/haptic technology (experimental group) for same hours respectively. To evaluate training effect, a national certificate test and a survey based on Phillips' ROI (Return on Investment) methodology were conducted by the students and the college respectively. And satisfaction survey among the students based on Kirkpatrick's Four-Level Evaluation Model was also carried out. The results showed that all students in the experimental group passed the national certificate test and the ROI of the experimental group for five years were 110% higher than the comparison group. Furthermore, 25% more students in the experimental group replied "very satisfied" about the overall training course and 75% more students in the same group found that the simulation was very similar to the real welding.