• Title, Summary, Keyword: Automated scoring

Search Result 36, Processing Time 0.036 seconds

Development of automated scoring system for English writing (영작문 자동 채점 시스템 개발 연구)

  • Jin, Kyung-Ae
    • English Language & Literature Teaching
    • /
    • v.13 no.1
    • /
    • pp.235-259
    • /
    • 2007
  • The purpose of the present study is to develop a prototype automated scoring system for English writing. The system was developed for scoring writings of Korean middle school students. In order to develop the automated scoring system, following procedures have been applied. First, review and analysis of established automated essay scoring systems in other countries have been accomplished. By doing so, we could get the guidance for development of a new sentence-level automated scoring system for Korean EFL students. Second, knowledge base such as lexicon, grammar and WordNet for natural language processing and error corpus of English writing of Korean middle school students were established. Error corpus was established through the paper and pencil test with 589 third year middle school students. This study provided suggestions for the successful introduction of an automated scoring system in Korea. The automated scoring system developed in this study should be continuously upgraded to improve the accuracy of the scoring system. Also, it is suggested to develop an automated scoring system being able to carry out evaluation of English essay, not only sentence-level evaluation. The system needs to be upgraded for the improved precision, but, it was a successful introduction of an sentence-level automated scoring system for English writing in Korea.

  • PDF

Automated Scoring System for Korean Short-Answer Questions Using Predictability and Unanimity (기계학습 분류기의 예측확률과 만장일치를 이용한 한국어 서답형 문항 자동채점 시스템)

  • Cheon, Min-Ah;Kim, Chang-Hyun;Kim, Jae-Hoon;Noh, Eun-Hee;Sung, Kyung-Hee;Song, Mi-Young
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.11
    • /
    • pp.527-534
    • /
    • 2016
  • The emergent information society requires the talent for creative thinking based on problem-solving skills and comprehensive thinking rather than simple memorization. Therefore, the Korean curriculum has also changed into the direction of the creative thinking through increasing short-answer questions that can determine the overall thinking of the students. However, their scoring results are a little bit inconsistency because scoring short-answer questions depends on the subjective scoring of human raters. In order to alleviate this point, an automated scoring system using a machine learning has been used as a scoring tool in overseas. Linguistically, Korean and English is totally different in the structure of the sentences. Thus, the automated scoring system used in English cannot be applied to Korean. In this paper, we introduce an automated scoring system for Korean short-answer questions using predictability and unanimity. We also verify the practicality of the automatic scoring system through the correlation coefficient between the results of the automated scoring system and those of human raters. In the experiment of this paper, the proposed system is evaluated for constructed-response items of Korean language, social studies, and science in the National Assessment of Educational Achievement. The analysis was used Pearson correlation coefficients and Kappa coefficient. Results of the experiment had showed a strong positive correlation with all the correlation coefficients at 0.7 or higher. Thus, the scoring results of the proposed scoring system are similar to those of human raters. Therefore, the automated scoring system should be found to be useful as a scoring tool.

Computerized Image Analysis of Micronucleated Reticulocytes in Mouse Bone Marrow (컴퓨터 이미지 분석법을 이용한 마우스 골수세포에서 소핵의 계수)

  • 권정;홍미영;고우석;정문구;이미가엘
    • Toxicological Research
    • /
    • v.18 no.4
    • /
    • pp.369-374
    • /
    • 2002
  • The present study was performed to validate an automated image analysis system (Loats Automated Micronucleus Scoring System) for the mouse bone marrow micronucleus assay, comparing with conventional microscopic scoring. Two studies were conducted to provide slides for a comparison of micro-nucleated polychromatic erythrocytes (MNPCEs) values collected manually to those collected by the auto-mated system. Test article A was used as an example of a compound negative for the induction of micronuclei and test article B was wed as a micronucleus-inducing agent to elicit a positive response. Cyclophosphamide was included to provide an positive control in two studies. Bone marrow samples were collected 24 h after administration of test article A and B in male ICR mice. The cells were fixed with absolute methanol and stained with May-Grunwald and Giemsa. The number of MNPCEs was determined by the analysis of 1000 total PCEs per bone marrow sample. In addition to micronucleus scoring, an index of bone marrow toxicity based on PCE ratio (% of PCEs to total erythrocytes) was determined for each sample. The automated and manual scoring was similar when the MNPCEs incidence induced by each test article was less than 10. However manual scoring was able to effectively enumerate micronucleated PCEs in mouse bone marrow when MNPCEs incidence was more than 10, such as cyclophosphamide treatment. Conversely, PCE ratio was superior in computer-assisted image analysis. Taken together, it is suggested that improvement of the automated image analysis may be necessary to render the automatic scoring as sensitive as manual scoring for routine counting of micronuclei, especially because it is superior in objectivity and high throughput scoring.

Scoring Korean Written Responses Using English-Based Automated Computer Scoring Models and Machine Translation: A Case of Natural Selection Concept Test (영어기반 컴퓨터자동채점모델과 기계번역을 활용한 서술형 한국어 응답 채점 -자연선택개념평가 사례-)

  • Ha, Minsu
    • Journal of The Korean Association For Science Education
    • /
    • v.36 no.3
    • /
    • pp.389-397
    • /
    • 2016
  • This study aims to test the efficacy of English-based automated computer scoring models and machine translation to score Korean college students' written responses on natural selection concept items. To this end, I collected 128 pre-service biology teachers' written responses on four-item instrument (total 512 written responses). The machine translation software (i.e., Google Translate) translated both original responses and spell-corrected responses. The presence/absence of five scientific ideas and three $na{\ddot{i}}ve$ ideas in both translated responses were judged by the automated computer scoring models (i.e., EvoGrader). The computer-scored results (4096 predictions) were compared with expert-scored results. The results illustrated that no significant differences in both average scores and statistical results using average scores was found between the computer-scored result and experts-scored result. The Pearson correlation coefficients of composite scores for each student between computer scoring and experts scoring were 0.848 for scientific ideas and 0.776 for $na{\ddot{i}}ve$ ideas. The inter-rater reliability indices (Cohen kappa) between computer scoring and experts scoring for linguistically simple concepts (e.g., variation, competition, and limited resources) were over 0.8. These findings reveal that the English-based automated computer scoring models and machine translation can be a promising method in scoring Korean college students' written responses on natural selection concept items.

A comparison of grammatical error detection techniques for an automated english scoring system

  • Lee, Songwook;Lee, Kong Joo
    • Journal of the Korean Society of Marine Engineering
    • /
    • v.37 no.7
    • /
    • pp.760-770
    • /
    • 2013
  • Detecting grammatical errors from a text is a long-history application. In this paper, we compare the performance of two grammatical error detection techniques, which are implemented as a sub-module of an automated English scoring system. One is to use a full syntactic parser, which has not only grammatical rules but also extra-grammatical rules in order to detect syntactic errors while paring. The other one is to use a finite state machine which can identify an error covering a small range of an input. In order to compare the two approaches, grammatical errors are divided into three parts; the first one is grammatical error that can be handled by both approaches, and the second one is errors that can be handled by only a full parser, and the last one is errors that can be done only in a finite state machine. By doing this, we can figure out the strength and the weakness of each approach. The evaluation results show that a full parsing approach can detect more errors than a finite state machine can, while the accuracy of the former is lower than that of the latter. We can conclude that a full parser is suitable for detecting grammatical errors with a long distance dependency, whereas a finite state machine works well on sentences with multiple grammatical errors.

Toward the Virtual Screening of α-Glucosidase Inhibitors with the Homology-Modeled Protein Structure

  • Park, Jung-Hum;Ko, Sung-Min;Park, Hwang-Seo
    • Bulletin of the Korean Chemical Society
    • /
    • v.29 no.5
    • /
    • pp.921-927
    • /
    • 2008
  • Discovery of $\alpha$-glucosidase inhibitors has been actively pursued with the aim to develop therapeutics for the treatment of diabetes and the other carbohydrate mediated diseases. As a method for the discovery of new novel inhibitors of $\alpha$-glucosidase, we have addressed the performance of the computer-aided drug design protocol involving the homology modeling of $\alpha$-glucosidase and the structure-based virtual screening with the two docking tools: FlexX and the automated and improved AutoDock implementing the effects of ligand solvation in the scoring function. The homology modeling of $\alpha$-glucosidase from baker’s yeast provides a high-quality 3-D structure enabling the structure-based inhibitor design. Of the two docking programs under consideration, AutoDock is found to be more accurate than FlexX in terms of scoring putative ligands to the extent of 5-fold enhancement of hit rate in database screening when 1% of database coverage is used as a cutoff. A detailed binding mode analysis of the known inhibitors shows that they can be stabilized in the active site of $\alpha$- glucosidase through the simultaneous establishment of the multiple hydrogen bond and hydrophobic interactions. The present study demonstrates the usefulness of the automated AutoDock program with the improved scoring function as a docking tool for virtual screening of new $\alpha$-glucosidase inhibitors as well as for binding mode analysis to elucidate the activities of known inhibitors.

Quality Assurance in Polysomnography - A Korean experience and critical suggestions (수면다원검사의 정도관리 - 한국에서의 경험 및 제언)

  • Jeong, Do-Un
    • Quality Improvement in Health Care
    • /
    • v.1 no.1
    • /
    • pp.124-131
    • /
    • 1994
  • Polysomnography is an essential methodology for diagnosing and following up sleep disorders and doing researches on human sleep. Sleep medicine, mainly with the utilization of polysomnographic techniques, has developed itself as one of the promising fields in the 21st century medicine. Korea is not an exception in importing and developing sleep medicine into the conventional medicine. However, it still remains to be clarified what polysomnography is for and how it should be done, considering the relatively recent introduction of sleep medicine into Korea. The author, being a board-certified sleep medicine specailist, having experienced spreading out sleep medicine within Korea for the past four years, and having recently set up a major sleep study facility in Korea at Seoul National University Hospital, attempts in this introductory critical article to review the essential issues related to quality assurance in polysomnographic study of human sleep. Also, unconditional introduction of "automated" sleep scoring system, which has been found to have significantly reduced reliability in various studies including the author's own, is critically reviewed. The author suggests that quality assurance and training program should be initiated and established by a responsible sleep medicine-related organization such as the Korean Association of Sleep Medicine and Psychophysiology.

  • PDF

Automated Pegboard Utilizing RFID System with Multiple Reader Antennas

  • Choi, Hyun-Ho;Ryu, Mun-Ho;Yang, Yoon-Seok;Shin, Yong-Il;Kim, Nam-Gyun
    • Journal of Biomedical Engineering Research
    • /
    • v.28 no.5
    • /
    • pp.585-589
    • /
    • 2007
  • This study proposes an automated pegboard utilizing the RFID system with multiple reader antennas for the rehabilitation services and the occupational therapy. The system automates the scoring by detecting the plugging correctness as well as the plugging status. It also aims to increase the patient's interest and the functional intelligence. The system was prototyped and tested for the automatic capability of the scoring the session time and success rate. The proposed system will be served as the typical example for the ubiquitous rehabilitation devices.

Accuracy Improvement of an Automated Scoring System through Removing Duplicately Reported Errors (영작문 자동 채점 시스템에서의 중복 보고 오류 제거를 통한 성능 향상)

  • Lee, Hyun-Ah;Kim, Jee-Eun;Lee, Kong-Joo
    • The KIPS Transactions:PartB
    • /
    • v.16B no.2
    • /
    • pp.173-180
    • /
    • 2009
  • The purpose of developing an automated scoring system for English composition is to score English writing tests and to give diagnostic feedback to the test-takers without human's efforts. The system developed through our research detects grammatical errors of a single sentence on morphological, syntactic and semantic stages, respectively, and those errors are calculated into the final score. The error detecting stages are independent from one another, which causes duplicating the identical errors with different labels at different stages. These duplicated errors become a hindering factor to calculating an accurate score. This paper presents a solution to detecting the duplicated errors and improving an accuracy in calculating the final score by eliminating one of the errors.

Implementing Automated English Error Detecting and Scoring System for Junior High School Students (중학생 영작문 실력 향상을 위한 자동 문법 채점 시스템 구축)

  • Kim, Jee-Eun;Lee, Kong-Joo
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.5
    • /
    • pp.36-46
    • /
    • 2007
  • This paper presents an automated English scoring system designed to help non-native speakers of English, Korean-speaking learners in particular. The system is developed to help the 3rd grade students in junior high school improve their English grammar skills. Without human's efforts, the system identifies grammar errors in English sentences, provides feedback on the detected errors, and scores the sentences. Detecting grammar errors in the system requires implementing a special type of rules in addition to the rules to parse grammatical sentences. Error production rules are implemented to analyze ungrammatical sentences and recognize syntactic errors. The rules are collected from the junior high school textbooks and real student test data. By firing those rules, the errors are detected followed by setting corresponding error flags, and the system continues the parsing process without a failure. As the final step of the process, the system scores the student sentences based on the errors detected. The system is evaluated with real English test data produced by the students and the answers provided by human teachers.