• Journal of Science Education
• /
• v.47 no.2
• /
• pp.211-224
• /
• 2023

The purpose of this study is to suggest how to investigate the reliability of the assessment, which consists of items generated by automatic item generation using empirical example data. To achieve this, we analyzed the illustrative assessment data by applying the multivariate generalizability theory, which can reflect the design of responding to different items for each student and multiple error sources in the assessment score. The result of the G-study showed that, in most designs, the student effect corresponding to the true score of the classical test theory was relatively large after residual effects. In addition, in the design where the content domain was fixed, the ranking of students did not change depending on the item types or items. Similarly, in the design where the item format was fixed, the difficulty showed little variation depending on the content domains. The result of the D-study indicated that the original assessment data achieved a sufficient level of reliability. It was also found that higher reliability than the original assessment data could be obtained by reducing the number of items in the content domains of operation, geometry, and probability and statistics, or by assigning higher weights to the domains of letters and formulas, and function. The efficient measurement conditions presented in this study are limited to the illustrative assessment data. However, the method applied in this study can be utilized to determine the reliability and to find efficient measurement conditions for the various assessment situations using automatic item generation based on measurement traits.

• Journal of Gifted/Talented Education
• /
• v.21 no.4
• /
• pp.861-884
• /
• 2011

In this study, we developed and applied 15 evaluation questions about light to 30 middle school students in K University Science Education Institute for the Gifted. Then we used the theory of knowledge space and analyzed the middle school Science Gifted Students' knowledge state about light. By schematizing the hierarchy from it, we intended to not only measure students' knowledge state about light, but use it as the basic materials to improve teaching methods. To achieve the purpose of this study, we analyzed the evaluation results and individual knowledge state and hierarchy of questions. As a result, there were different knowledge structures in the individual, and we found that we should diagnose them differently. In addition, we have had implications that it has the connection with each questions and the individual knowledge state.

• Journal of Korean Society of Forest Science
• /
• v.111 no.3
• /
• pp.450-460
• /
• 2022

This study was conducted to develop a forest ecological sensitivity scale and verify its reliability and validity to evaluate the effectiveness of forest education programs and assess the forest ecological sensitivity of people. Forest ecological sensitivity was defined through literature research and expert meetings. A total of 106 preliminary questions, which were developed by dividing the factors of each question, were used to conduct a preliminary survey on 256 adults. Based on the survey results, an exploratory factor analysis was conducted, and 19 questions were selected, excluding overlapping items between factors and items not suitable for factor load. This was a nationwide study in which 557 adult individuals participated, and 533 surveys were used for analysis. According to the results of the exploratory and confirmatory factor analysis, the forest ecological sensitivity scale consisted of 10 questions and 3 factors (personal stage, social stage, and integrated stage) and demonstrated good suitability. The forest ecological sensitivity questions developed in this study can be used to evaluate the effectiveness of forest education programs, which can help contribute toward improving the quality of such programs.

• The Journal of the Korea Contents Association
• /
• v.17 no.10
• /
• pp.472-482
• /
• 2017

In this study, discrete mathematical items were classified into analytical items and mathematical items were analyzed on the basis of analytic framework items of mathematics and the past items of mathematics subject contents of the period 2011-2017 school year. First, the discrete mathematics evaluation areas and evaluation contents proposed by the Korea Institute for Curriculum and Evaluation should be evenly distributed. Second, the items of measuring metacognitive knowledge as a strategic knowledge on the use of cognitive methods should be given. Third, the ratio of the number of items in discrete mathematics to the number of that was 3.8%~6.8%, and the ratio according to the item weighting was 2.2%~6.3%. Fourth, it is analyzed that all the items are suitable for the evaluation goal and the pre-service math teachers who have faithfully implemented the curriculum have maintained the appropriate level of difficulty to solve. Finally, the content items such as the method of counting the discrete mathematics curriculum, the Recurrence Relation, the generation function, and the graph are matched with the teacher certification examination and the mathematics education curriculum of each teachers college. By these reasons, we conclude that the contribution of pre-service teachers to the motivation of learning is obtained and implications.

• Journal of the Korean School Mathematics Society
• /
• v.20 no.3
• /
• pp.325-344
• /
• 2017

Since PISA2006, the computer based assessment in mathematics(CBAM) was introduced for the first times and at last PISA2015 used all items in CBAM for problem solving. In this study, we focused on which important properties were considered in constructing geometric 'fence items' used in PISA 2015 to find the future direction over our teacher education, especially for constructing 'computer based assessment items.' For the purpose of the study, we analyzed the fence items on three components such as dependency, invariant, and path found in dragging activities, within a computer environment using the dynamic Geometry Software, GSP. Also, for the future, we provided an open-ended problem related to the fence items, which we could use as the merit of computer-based environment.

• Journal of Educational Research in Mathematics
• /
• v.25 no.2
• /
• pp.157-175
• /
• 2015

This study was conducted with the aim to develop and improve the assessment literacy of teachers which enables it to develop and utilize the assessment items using a wide range of contexts on the basis of an understanding of math contents and mathematical process specified in the math curriculum and to analyze the results in an effective way. To analyze the changes in the development and improvement of the assessment literacy of preservice math teachers, the author of this study, using PISA items and assessment framework, analyzed the changes through the 1st and 2nd development of assessment items. The results showed the assessment literacy of preservice math teachers and their ability have been improved, implying that there is the necessity to develop a wide range of programs to improve the assessment literacy of preservice math teachers and provide such programs on a regular basis, which will facilitate effective math teaching and learning.

• The Journal of Korean Association of Computer Education
• /
• v.21 no.4
• /
• pp.29-37
• /
• 2018

In this paper, questions from the last three years' appointment examinations for secondary teachers for informatics computer subjects are classified in a way to fit thirteen basic mandatory courses, for which the composition of scores and the number of questions tested were examined. Of these thirteen basic mandatory courses, questions from six basic mandatory courses with high frequency of assessment were analyzed on their score composition based on the evaluation area. In addition, questions testing knowledge in C language were analyzed separately. The results show scores for the curriculum content subjects ranged from 0% to 17.1%, and the ratio of questions tested from curriculum content subjects 0% to 15.8%. In addition, questions from three courses showed a large deviation in scores for each year. Second, the score composition based on the evaluation area showed a large discrepancy, ranging from 0% to 52.0%. Lastly, the questions tested on the C language exhibited high score on average at 33%. Based on the results from the analysis, this study proposed plans to improve ways to evaluate test takers' aptitude.

• Journal of Science Education
• /
• v.37 no.3
• /
• pp.458-475
• /
• 2013

According to instruction of standard-based assessment system, teachers with higher level of competence in student assessment are required. The purpose of this study is to offer the basic data to enhance pre-service physics teachers' competency in student assessment. 17 pre-service physics teachers, who are sophomore in department of physics education, participated in the study for a whole semester. To do this, we gave the assignment of multiple choice question construction in physics conceptest for formative assessment to pre-service teachers and observed review questions constructed by them. We also analyzed multiple choice questions of 17 pre-service teachers. They had 5 classes for the review process of the question and improved their questions through it. 16 of 17 pre-service teachers completed test design. Our results show that pre-service teachers had a little low competence in test design for classroom assessment. In addition to, constructing and revising questions, almost of them didn't have various perspectives and didn't consider to get information about the pre-conceptions of students. These findings indicate that they have difficulties with constructing multiple choice question and more specific plan needs to enhance the overall student-assessment competency levels of pre-service teachers for successful settlement of the standard-based assessment system.

• The Journal of the Korea Contents Association
• /
• v.10 no.7
• /
• pp.438-448
• /
• 2010

This study is aimed for a scale of assessing the work performance, educational operations, and so on of preschool teachers by the peers. It has been previously discussed the studies of scale assesment of preschool teachers, however, the scale assesment had been a self-assesment or a transitory assesment by the senior teachers and principals only. This study is developed to assess the preschool teachers through the peer- teachers. Peer-assesment has strong points that to assess the teachers constantly can be more accurate, the assessment result can be used as the basic material for peer scholarship, and it can give the preschool teachers an opportunity to be recognised on tasks. There are 51 questions are developed to assess the peers. 9 sections total; 1) 6 questions on body human nature 2) 6 questions on view on career 3) 3 questions on self-development 4) 4 questions on curriculum planing 5) 12 questions on administering and managing 6) 6 questions on understanding children 7) 4 questions on managing a class 8) 7 questions on work performance 9) 3 questions on teaching practice. Each section is surveyed by 150 preschool teachers, and the all questions are analysed statistically appropriate of a scale(validity of contents, validity of surveyors, inter-item consistency) and appropriate of each section. It is known this peer assesment of preschool teachers is an appropriate scale.

• School Mathematics
• /
• v.19 no.1
• /
• pp.43-58
• /
• 2017

According to the degree of teacher's understanding for the curriculum, There are a lot of differences in teaching-learning methods and assessment items are one of the representative products reflecting these differences. Therefore we need to investigate how the understanding degree of the teacher for curriculum is reflected in the paper-based assessment items through analyzing them. In this study, we analyzed midterm and final 219 exam papers of 'Calculus I' which was based on the 2009 revised mathematics curriculum and focused on items of 'the Limit of Sequences' which content area is among total 4,632 questions. We investigated how the changed curriculum is reflected in the high school evaluation. Based on the results of the analysis, we confirmed the problems derived from the paper-based assessment. Through this, we sought to draw implications for the educational policy that should be accompanied necessarily in order to stabilize the new curriculum after the revision of the curriculum.