The Efficiency of Objective Structured Clinical Examination (OSCE) as a Method of Clinical Clerkship in the Korean Ophthalmology & Otolaryngology Department

한방안이비인후과에서 임상실습 한 방법으로서 Objective Structured Clinical Examination (OSCE)의 효용성

Abstract

Objectives : 본 연구를 통하여 한방안이비인후과 임상실습에서 Objective Structured Clinical Examination (OSCE) 수업의 효용성을 알아보고자 한다. Methods : 2014학년도 한의과대학 본과 4학년 학생을 대상으로 안이비인후과 임상실습의 한 방법으로 OSCE를 시행한 후, 학생들은 OSCE 수업 만족도에 관한 설문지를 작성하고 레지던트는 학생들을 평가하였다. OSCE 평가 성적과 기타 임상실습 및 필기시험 성적과의 연관성을 분석하여 OSCE 성적의 학점 부여 타당도를 살펴보았다. Results : 만족도 관련 설문 결과 총 평균점수가 4.34점이 나왔다. OSCE성적과 기타 임상실습 및 필기시험 성적과의 상관성을 pearson correlation을 통하여 살펴본 결과, OSCE성적은 임상실습 성적 및 (r=0.210, p=0.056), 필기시험 성적과 통계적으로 유의한 상관성이 없었다 (r=0.064, p=0.567). Conclusions : 위의 결과로부터 임상실습에서 OSCE 시행은 학생들의 학습 만족도를 높인다는 것을 알 수 있었다. 임상실습에서 효율적으로 OSCE를 활용하는 방안을 모색하기 위해 더 많은 연구가 필요할 것으로 생각된다.

Ⅰ. Introduction

The objective of medical education for medical school students is not only to have sufficient technical knowledge but also to make graduates capable of performing as a doctor professionally and independently, meaning to being equipped with ability to collect information, analyzing symptoms, and conduct medical treatment accordingly based on medical related knowledge1-3). In order to assess capability of students as to whether they can apply it in a real situation, skills and attitudes as well as medical knowledge should be put on evaluation and in order to do this, Objective Structured Clinical Examination (OSCE) has been implemented to domestic medical board examination since 2010. OSCE is one of methods to estimate actual skills, which is to see how students handle particular situation that is similar to real case and test on skills4).

However, there has been an increasing number of thesis which regards OSCE not only as an evaluation vehicle but also as an application to improve students’ ability of practicing medical treatment. It is reported that students those who had OSCE before a clinical training has higher competence than those who did have OSCE after a clinical training in terms of conducting physical examination5). In addition to that, it is also reported that students who went through OSCE test has significantly higher score of cohort in terms of conversational skill than those who did not6). Hence, domestic medical schools have been attempting to bring in OSCE for a clinical training aside from an evaluation tool for a clinical training.

Most Korean medical schools have been utilizing assessment of clinical clerkship, focusing on acquiring background knowledge through reports, presentation, and written examination and some schools have already adopted OSCE for a clinical clerkship, but they are lacking in research.

Thus, the department of Ophthalmology & Otolaryngology at the Korean medical school of Dongguk university has examined student’s satisfaction level and pros and cons of a clinical clerkship which applied OSCE through the survey conducted right after students experience a clinical clerkship with OSCE. Also, by comparing the relevance between OSCE and other examinations, we attempt to navigate ways to make use of OSCE effectively in a clinical clerkship.

 

Ⅱ. Subject and Method

In 2014, in the clinical clerkship coursework of Ophthalmology & Otolaryngology department, 83 seniors in the college of Korean medicine had participated in OSCE class, composing a group of 6-7 students. Eye Examination Simulator and Ear Examination which were purchased from Kyoto Kagaku Co. were used as an OSCE model. It started with the lecture by a resident on basic background knowledge about eye and ear related diseases (normal, hypertensive retinopathy, diabetic retinopathy, acute and chronic papilloedema, glaucomatous optic atrophy, retinal vein occlusion, toxoplasmosis, age-related macular degeneration/ normal, serous otitis media, mucoid otitis media, chronic suppurative otitis media, acute suppurative otitis media, cholesteatoma, tympanosclerosis, traumatic perforations, cerumen block) that could be observed by each stimulator and explanation about how to use ophthalmoscope and otoscope. A few days later, students had spent some time on learning how to diagnose eye and ear related diseases and how to handle ophthalmoscope and otoscope, using actual simulators by themselves. After that, the resident evaluated students overall skill sets (proper use of ophthalmoscope and otoscope, appropriate attitudes in terms of using simulator, ability to diagnose disease correct, etc.) and thses evaluation items are same as an existing evaluation tool which is conducted in the actual OSCE part of the medical board examination. And then students had conducted surveys (Table 1). When the clinical clerkship was finished, we took a look at validity of granting scores to OSCE results, taking the relevance of OSCE evaluation result, other results from a clinical clerkship (case report, presentation), and written exam taken by seniors into consideration.

Table 1.Questionnaire

The survey comprised of multiple choice questions, referred by preceded domestic researches related to western medical school’s clinical clerkship and evaluation method7-9). We set the score on a scale of 1 to 5, using Liker’s 5-stage rating scale, which is Strongly agree (5), Agree (4), Neither agree nor disagree (3), Disagree (2), Strongly disagree (1). The collected data was processed by SPSS 12.0 for Windows and Cronbach’s α was sought for item reliability, frequency and percentage were sought to see the general traits of research participants, and the mean, standard deviation, frequency of response, and percentage were sought to see the satisfaction of a clinical clerkship. The Pearson’s correlation coefficient was calculated to identify the relevance of OSCE result and other result.

 

Ⅲ. Result

A. General trait of research objectives

The reliability coefficient (Cronbach’s α) of the total 20 questions turned out to be 0.917, resulting from analyzing reliability of questionnaire. The general trait of research objective was that the proportion of males to females among 83 students was 54.2% (45 males) to 45.8% (38 females). Also, the age ranging 21-25 years old were 35 students (42.2%), 26-30 years old were 36 students (43.4%), and above 31 years old were 12 students (14.5%) (Table 2).

Table 2.General Characteristics

B. Satisfaction of clerkship, using OSCE

As a result of conducting satisfaction survey, which consists of 20 questions, the mean and the standard deviation for each question were as follows (Table 3). The total mean was 4.34 and for the satisfaction of measuring method (Q1-12), the mean was 4.35 and for effectiveness (Q13-20), it was 4.31.

Table 3.Satisfaction Scores of Objective Structured Clinical Examination (OSCE)

1. Satisfaction as a clinical training method

We have conducted survey of 12 questions in terms of interest level and usefulness of OSCE as one method of clinical clerkship and the number of respondents and response ratio for each question are as follows (Table 4).

Table 4.Number and Percentage of Satisfaction Score

2. Satisfaction on usefulness

8 questions were about the efficiency of the OSCE as an educational method. The number of respondents and ratio of responses for each question are as follow (Table 4).

3. Studentʼs opinion on operation of OSCE

We surveyed students’ opinions on adequate frequency, appropriate year of grade and the number of courses to implement OSCE and the results came out as follows (Table 5) .

Table 5.A Student’s Opinion on Operation of Objective Structured Clinical Examination (OSCE)

4. Relevance among the grade in OSCE, clinical clerkship and academic record

We have compared and analyzed the relevance among the grade in OSCE, clinical clerkship and academic record through the Pearson correlation (Table 6). There was no statistically significant relevance between the grade in OSCE and clinical clerkship for the 1st semester of senior year (r=0.210,p=0.056). And this also applies to the academic performance of the 1st semester of senior year (r=0.064,p=0.567).

Table 6.*p<0.05

 

Ⅳ. Discussion & Conclusion

OSCE was proved to be reliable and valid as one measuring method of clinical performance ability through a variety of researches and has been used in developed countries and domestic medical schools as an important evaluation vehicle for clinical skills. Furthermore, OSCE has been recognized as a useful learning tool in that it could advance students’ performance ability by its repetitiveness and feedback. Thus, increasing number of medical school is utilizing OSCE as a learning tool7,10,11).

We have conducted a clinical training, using OSCE, targeting at the seniors in the department of Ophthalmology & Otolaryngology at the Korean medical school of Dongguk university and after that we have surveyed on student’s satisfaction and opinion on the future direction of training through questionnaire. We have taken a look at the relevance between OSCE evaluation result and other test results as well.

First of all, the survey has an inner conformity as a research method as the reliability coefficient was 0.917 in the total 20 questions of satisfaction items. 83 seniors participated in the clinical clerkship and they were respondents to the questionnaire as well. There were 45 male students (54.2%) and 38 female students (45.8%) and the age distributions was that 35 students (42.2%) were between 21-25 years old, 36 students (43.4%) were between 26-30 years old, and 12 students (14.5%) were above 31 years old. As a result of analyzing survey, student’s satisfaction level was high, given that the mean of the total questions was 4.34 when it comes to satisfaction of clinical clerkship. The survey questions can be grouped into two agendum, which are ‘method satisfaction’(Q1-12), asking interest and appropriateness of OSCE as a learning method and ‘effectiveness’(Q13-20), asking educational effectiveness of OSCE. For the ‘method satisfaction’, the mean was 4.35 and for the ‘effectiveness’, the mean was 4.31. The satisfaction on the new clinical clerkship method, using OSCE seemed high, considering both mean turned out to be high.

Q7 ‘I agree that juniors have a chance of using OSCE during a clinical training class.’ has the highest mean of 4.72 among ‘method satisfaction’ related questions. Q2 ‘I actively participated in the training class.’ and Q4 ‘I think that it is appropriate to implement OSCE into a clinical training class.’ have the second highest mean of 4.64. The lowest mean of 3.82 goes to Q8 ‘The level of difficulty of OSCE was reasonable.’ and Q 9 ‘The duration time for a training class with OSCE was sufficient’. This could be interpreted as students believe that OSCE was a proper learning method for a clinical clerkship as a result of they actually had conducted OSCE, but the level of difficulty and duration time need to be adjusted.

In terms of effectiveness of OSCE in the clinical clerkship, the survey consists of knowledge, skills, and general question parts. Q18 ‘It was helpful to use OSCE in a clinical training class for improvement of method of otoscopy.’ and Q17 ‘It was helpful to use OSCE in a clinical training class for improvement of method of direct ophthalmoscopy.’ have the high mean of 4.48 and 4.39 respectively and Q 19 ‘I become more confident on my practical skills after having experience with OSCE in a clinical training class.’ has the lowest mean of 4.08. These results reflect the fact that OSCE was an effective method to practice otoscope and ophthalmoscope in an actual environment which students had only learnt from text books but the number of practicing OSCE needs to be increased as a one-time practice is not enough to enhance student’s confidence.

As a result of survey about student’s satisfaction, there was a high portion of questions that have 3-5 point of answers and this shows that students have high satisfaction about clinical clerkship, using OSCE and they regard it as an interesting method of learning in a class.

Concerning the operation of OSCE, 32.5% of students responded that adequate frequency of implementing OSCE was twice and 56.6% chose to implement OSCE more than twice. Also, for proper timing of using OSCE, 15.9% of students voted for junior year. 19.5% voted for senior year, and 64.6% voted for both junior and senior years. In terms of the number of subjects using OSCE, 28.9% of students answered that the number of subjects should be increased from the current three courses and 37.5% of students answered that all subjects need to implement OSCE. Given these results, we could observe that students are satisfied with practicing clinical skills through OSCE and they think it as a necessary learning course.

We had collected pros and cons and improvements of OSCE training and the major opinions were such as, ‘It was helpful to practice actual method skills rather than learning from a textbook.’, ‘It was good to have an experience with various diseases.’, ‘It would be nice if we could observe frequently when it is necessary, not just in a clinical training course.’, ‘We could get to know about which part we need to make for what they lack in a class.’ and these answers are telling that student’s satisfaction has risen by having an opportunity of practice clinical skills proactively, not passively. Also, the OSCE helped students to be motivated to study hard in that they could well recognize their own status of clinical skill ability and improvements to be made. Many students also said, ‘Having a sufficient practice experience before implementing methods to actual patients was helpful.’ which means that OSCE can bridge a gap of theory and practice as it enables students to treat patients without stress or embarrassment in an actual situation through regular and repetitive practices. Students pointed out weaknesses of OSCE as follows, ‘Somehow there was a gap from an actual patient.’, ‘At the first place, it was hard to observe. The difficulty level needs to be adjusted.’, ‘When it comes to otoscopia, there were times that strange substances were found.’ Meaning that a diversity of difficulty levels needs to be developed and a continuous maintenance for the optimal condition of OSCE is essential.

In terms of relevance between grade in OSCE and grades in clinical clerkship and academic performance, there was no statistically significant relevance among them. It could be interpreted as OSCE grade does not have validity but can be seen as OSCE evaluated other facets that existing test could not. In other words, OSCE would be able to be utilized as a supplementary student evaluation tool for essential skills and attitudes which a doctor is required to possess, and hard to be assessed by existing written or clinical training examination12-15).

As we put all the results from this study together, we could see from students’ survey responds and opinions described that students had interestingly participated in an OSCE class which was one new way of clinical trainings and positively regarded it as necessary.

However, this research needs improvements in that its target objects were only limited to a small number of 83 students from one Korean medical school senior year, it was not repetitively conducted due to a short period of education time, and a student’s level of learning ability was not considered. Thus, in order to estimate usefulness of OSCE precisely, the research target objective should be expanded into other grades and other universities as well and clinical training class, implementing OSCE should be conducted over years so that we could collect data from accumulated years of experience and the OSCE class should be compared with other clinical training programs.

Additionally, OSCE is strong in evaluating clinical performance ability objectively for a short period of time but limited in observing the relationship between a patient and a doctor to some extent. Therefore, OSCE would be able to provide students with a clinical situation closer to reality if it is not confined to a fragmentary clinical skill training but also it is extendedly applied into a way of practicing an extensive problem solving ability and assessment, including basic counseling skills based on case development of patient’s current disease history and family disease history, medical checkup using OSCE, interpretation of clinical data, diagnosis, and plan for treatment. Recently one problem found in difference between real patients and simulated patient has been brought up in Problem based learning (PBL), Standardized Patients (SP) and Role play which are implemented as a method of clinical training and if we could adopt OSCE model then the gap would possibly reduced. Also, giving a feedback such as scores for an each item and comments after an evaluation would better enhance learning performance.

This research was meaningful in that it offers a stepping stone of increasing student’s learning satisfaction and ability of clinical performance through OSCE in a clinical clerkship. Also, considering the fact that a report on a clinical training class, using OSCE and observing its effectiveness at a Korean medical school is rarely found, the limitation, and improvements presented by this research would be a basic reference to make a plan for clinical training curriculum. OSCE can function as teaching and learning tool as well as an evaluation vehicle so if the current clinical training education system would be improved according to research and development of various items and methods, then better clinical training education system which can enhance practical skills required for an actual clinical situation and counseling ability would be achieved.