• English Language & Literature Teaching
• /
• v.7 no.2
• /
• pp.17-36
• /
• 2002

This article begins by exploring recent developments in the use of the world wide webs in language testing about what a Web-based language test (WBT) is and how they are used in language testing. After a brief review of computer-based testing, WBTs are defined and categorized as low-tech or high tech. Since low-tech tests are the more feasible, they will constitute the focus of this paper. Next, item types for low-tech WBTs are described, and validation concerns that are specific to WBTs are discussed. After a brief overview of the combination of computer-adaptive and Web-based tests, the general advantages as well as design and implementation issues of WBTs are considered before examining the role that testing consequences play in deciding whether a WBT is an appropriate assessment instrument. It is argued that WBTs are most appropriate in low-stakes testing situations; but with proper supervision, they can also be used in medium-stakes situations although they are not generally recommended for high-stakes situations. Some possible ideas for future research are suggested.(169)

• English Language & Literature Teaching
• /
• v.15 no.4
• /
• pp.157-175
• /
• 2009

This paper offers a critical review of a newspaper argument regarding the problems of high school education assessment for university entrance examination system in the United Kingdom. The media account raises three sets of questioning to hold that the nation's long-standing A-levels have failed and is no longer viable as a high-stakes test. However, it is found that the writer's argumentations involving misleading conceptions can be deconstructed because of invalid reasoning and unreliable evidence. So, it is proposed that a reasonable solution to replace the discredited A-level exams should be to adopt an eclectic approach for assessing candidates' multiple capabilities; performance, potentiality and critical thinking skills. These criteria for component-oriented assessments are designed to measure their high school academic achievements and intellectual capacity for tertiary education; in the process of such measurement, critical-logical reasoning abilities for sound judgment and problem-solving tasks should be incorporated with the basic precondition that each university possesses its own discretion for the determination of adequate proportions to reflect each of the assessment outcomes. It is, therefore, expected that this critical review will inspire the readers to understand aspects of assessment as an educational field and to confirm how seriously they may be misguided by a distorted media argumentation without substantive background knowledge.

• English Language & Literature Teaching
• /
• v.18 no.2
• /
• pp.189-211
• /
• 2012

The purpose of the study was to explore ways to improve the test items on the Korean scholastic ability test. More specifically, the researchers investigated whether use of the target language in test items would make a difference in total scores, discriminatory power, and item difficulty. A total of 288 high school seniors participated in the study. The subjects were divided into the experimental group (N=145) and the control group (N=143). A 25-item test resembling the Korean scholastic ability test was administered to both groups. The experimental group was given items whose questions and alternatives were all presented in English, whereas the control group was given items whose questions and alternatives were presented in Korean only. Statistical analyses revealed that use of English vs. Korean in the questions and alternatives made a significant difference in total scores, item discrimination, and item difficulty level. The findings strongly suggest that use of English is one way to improve the quality of the Korean scholastic ability test by enhancing item discrimination and face validity. Considering that the test in question is a high-stakes exam in Korea, further research on how to improve the Korean scholastic ability test is urgently called for.

• Communications of Mathematical Education
• /
• v.28 no.2
• /
• pp.195-217
• /
• 2014

The College Scholastic Ability Test(CSAT) is the Korean national university examination based on the national curriculum. The CSAT is a high-stakes test because of powerful social forces which the college admission system has in Korea. This examination has changed many times through not only the national curriculum revision but also various external factors including the normalization of public education, mitigating the burden of students, etc. This study analysis the changes of assessment contents of the Mathematics of the CSAT due to the national curriculum revision. Additionally, this study analysis the mathematics content domains of the college entrance examinations in some foreign countries. Based on the result of this analysis, this study will derive implications for improvement directions of the Mathematics of the CSAT.

• The Journal of Korean Medicine
• /
• v.45 no.1
• /
• pp.1-16
• /
• 2024

Introduction: The importance of clinical skills training in traditional Korean medicine education is increasingly emphasized. Since the clinical skills tests are high-stakes tests that determine success in national licensing exams, it is essential to develop reliable multifaceted analysis methods for clinical skills tests in actual education settings. In this study, we applied the multifaceted validity evaluation methods to the evaluation results of the cardiopulmonary resuscitation module to confirm the applicability and effectiveness of the methods. Methods: In this study, we used internal consistency, factor analysis, generalizability theory G-study and D-study, ANOVA, Kendall's tau, descriptive statistics, and other statistical methods to analyze the multidimensional validity of a cardiopulmonary resuscitation test in clinical education settings over the past three years. Results: The factor analysis and internal consistency analysis showed that the evaluation rubric had an unstable structure and low concordance. The G-study showed that the error of the clinical skills assessment was large due to the evaluator and unexpected errors. The D-study showed that the variance error of the evaluator should be significantly reduced to validate the evaluation. The ANOVA and Kendall's tau confirmed that evaluator heterogeneity was a problem. Discussion and Conclusion: Clinical skills tests should be continuously evaluated and managed for validity in two steps of pre-production and actual implementation. This study has presented specific methods for analyzing the validity of clinical skills training and testing in actual education settings. This study would contribute to the foundation for competency-based evidence-based education in practical clinical training.

• The Journal of Korean Association of Computer Education
• /
• v.17 no.2
• /
• pp.39-51
• /
• 2014

In high-stakes tests for large groups, the efficiency with which students' responses are distributed to raters and how systematic scoring procedures are managed is important to the overall success of the testing program. In the scoring of constructed response items, it is important to understand whether the raters themselves are making consistent judgments on the responses, and whether these judgments are similar across all raters in order to establish measures of rater reliability. The purpose of this study was to design, develop and carry out a pilot test of an online scoring system for constructed response items administered in a paper-and-pencil test to large groups, and to verify the system's reliability. In this study, we show that this online system provided information on the scoring process of individual raters, including intra-rater and inter-rater consistency, compared to conventional scoring methods. We found this system to be especially effective for obtaining reliable and valid scores for constructed response items.

• Journal of Gifted/Talented Education
• /
• v.17 no.2
• /
• pp.392-412
• /
• 2007

It has been understood that Confucian culture only blocks creative problem solving(CPS). However, it is claimed that Confucian culture exerts positive and negative influence on nurturing CPS. Confucian culture fosters CPS by urging strong motivation for learning, and building rich base of knowledge and skills in general and in specific domains through teaching joy of learning and hardworking fueled by parents' education fever. Confucian culture has negative influence on CPS with emphasis on obedience and conformity. Keju system, as a part of Confucian culture, resulted in excessive private tutoring to prepare for the high-stakes test, delimitation of students' experiences to matters to be tested, and students' weak interest, self-concept, and self-directed learning. Negative influence from Confucian culture on CPS is analyzed and some suggestions are provided for revising educational policies and educational practices in order to maximize the creative problem solving in Confucian countries.