• Title/Summary/Keyword: Consistency in Assessment

Search Result 333, Processing Time 0.024 seconds

Semi-Automatic Scoring for Short Korean Free-Text Responses Using Semi-Supervised Learning (준지도학습 방법을 이용한 한국어 서답형 문항 반자동 채점)

  • Cheon, Min-Ah;Seo, Hyeong-Won;Kim, Jae-Hoon;Noh, Eun-Hee;Sung, Kyung-Hee;Lim, EunYoung
    • Korean Journal of Cognitive Science
    • /
    • v.26 no.2
    • /
    • pp.147-165
    • /
    • 2015
  • Through short-answer questions, we can reflect the depth of students' understanding and higher-order thinking skills. Scoring for short-answer questions may take long time and may be an issue on consistency of grading. To alleviate such the suffering, automated scoring systems are widely used in Europe and America, but are in the initial stage in research in Korea. In this paper, we propose a semi-automatic scoring system for short Korean free-text responses using semi-supervised learning. First of all, based on the similarity score between students' answers and model answers, the proposed system grades students' answers and the scored answers with high reliability have been included in the model answers through the thorough test. This process repeats until all answers are scored. The proposed system is used experimentally in Korean and social studies in Nationwide Scholastic Achievement Test. We have confirmed that the processing time and the consistency of grades are promisingly improved. Using the system, various assessment methods have got to be developed and comparative studies need to be performed before applying to school fields.

Translation and Adaptation of the Children's Home Inventory for Listening Difficulties (CHILD) into Korean (가정환경 아동듣기평가(CHILD) 부모용 설문지의 한국어 번역 및 적용 연구)

  • Choi, Jae Hee;Seo, Young Ran;Jang, Hyun Sook
    • 재활복지
    • /
    • v.20 no.4
    • /
    • pp.247-264
    • /
    • 2016
  • The Children's Home Inventory for Listening Difficulties (CHILD) questionnaire has been applied for assessing listening and communication difficulties in various home situations for children with hearing loss. The purpose of the study was to translate the CHILD questionnaire for parents into Korean and verify reliability and validity of Korean version of CHILD (CHILD-K). CHILD-K was completed by 55 parents of children (from ages 3~12 years) using cochlear implants (CI). Among the 55 children, 27 were in preschool and 28 in elementary. Internal consistency reliability of CHILD-K was verified by Chronbach's alpha. The mixed factorial ANOVA was conducted to compare the effects of the age group and situation factors (Quiet, Noise, Distance, Social, and Media factors) on the score of CHILD. The results indicated that CHILD-K showed excellent internal consistency reliability (${\alpha}=.96$). The CHILD scores among age groups were significantly different as the older age group resulted in higher scores in all situations except Distance. For both groups the mean scores for the Quiet situation were significantly higher than other situations, and the mean scores for the Social situation were significantly lower than other situations. Moreover, analysis showed that children with CI had difficulties in the Social situation combined with other situation factors. The results indicate that the Korean version of CHILD questionnaire is a reliable tool for the assessment of communication abilities in home situation in Korean-speaking children using CI.

Spatial Clustering Analysis based on Text Mining of Location-Based Social Media Data (위치기반 소셜 미디어 데이터의 텍스트 마이닝 기반 공간적 클러스터링 분석 연구)

  • Park, Woo Jin;Yu, Ki Yun
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.23 no.2
    • /
    • pp.89-96
    • /
    • 2015
  • Location-based social media data have high potential to be used in various area such as big data, location based services and so on. In this study, we applied a series of analysis methodology to figure out how the important keywords in location-based social media are spatially distributed by analyzing text information. For this purpose, we collected tweet data with geo-tag in Gangnam district and its environs in Seoul for a month of August 2013. From this tweet data, principle keywords are extracted. Among these, keywords of three categories such as food, entertainment and work and study are selected and classified by category. The spatial clustering is conducted to the tweet data which contains keywords in each category. Clusters of each category are compared with buildings and benchmark POIs in the same position. As a result of comparison, clusters of food category showed high consistency with commercial areas of large scale. Clusters of entertainment category corresponded with theaters and sports complex. Clusters of work and study showed high consistency with areas where private institutes and office buildings are concentrated.

The Comparative Quantitative Risk Assessment of LNG Tank Designs for the Safety Improvement of Above Ground Membrane Tank (지상식 멤브레인 LNG저장탱크 안전성 향상을 위한 설계형식별 정량적 위험성 비교 평가)

  • Lee S.R.;Kwon B.G.;Lee S.H.
    • Journal of the Korean Institute of Gas
    • /
    • v.9 no.4 s.29
    • /
    • pp.57-61
    • /
    • 2005
  • The objective of paper is to carry out a comparative Quantitative Risk Assessment (QRA) of two KOGAS tank designs using a fault tree methodology, a standard 'Full Containment' tank and a 'Membrane' tank. For the membrane tank, both the initial KOGAS design and 4 modified KOGAS designs have been assessed, giving six separate cases. In this paper, the frequencies of releases are quantified using a fault tree approach. For clarity in the analysis, and to ensure consistency, all cases have been quantified using the same fault tree. Logic within the fault tree is used to select each of the cases. Full quantification of risks is often difficult, owing to a lack of relevant failure data, but the aim of this study has been to be as quantitative as possible, with full transparency of failure information. The most significant general cause of external LNG leaks is predicted to be a seismic event, which has been quantified nominally. 4modified KOGAS desiens to Prevent damage of bottom membrane panels that was shown in preparatory estimation could quantitively confirm safety improvement. According to result, the predicted frequencies of an external LNG leak for the full containment and modified membrane tanks are very similar, failures due to dropped pumps are predicted to be significantly greater for the membrane tank with thickened plate than for the full containment tank.

  • PDF

A Study on Validity, Reliability and Practicality of a Concept Map as an Assessment Tool of Biology Concept Understandings (생물 개념 이해의 평가 도구로서 개념도의 타당도, 신뢰도 그리고 현실 적용 가능성에 대한 연구)

  • Cho, Jung-II;Kim, Jung
    • Journal of The Korean Association For Science Education
    • /
    • v.22 no.2
    • /
    • pp.398-409
    • /
    • 2002
  • The purpose of this study was to investigate the validity, reliability and practicality of a concept map as an assessment tool in the context of biology concept learning. Forty undergraduate students participated in concept mapping, and the maps were scored by preservice science teachers, using one of three different scoring methods, that is, concept map scoring methods developed by Burry-Stock, Novak & Gowin and McClure & Bell. Two scorers were assigned to each scoring method. As far as the validity of the assessment methods was concerned, two of the three methods were found to be very valid, while Burry-Stock's scoring method was shown little valid. As far as the internal consistency of the methods was concerned, considerably high consistencies were shown between every pair of scorers, judging from high correlation coefficients between the pair of scorers for each scoring method. It took from 1.13 minutes to 3.70 minutes to assess a map at the average. It showed that concept mapping could be used in school classrooms with the limited resources of time and people. These findings suggest that the concept mapping can be an appropriate tool for assessing biology concept understandings.

An Assessment of the Validity of the Marital Satisfaction Scale -Mss of Roach , Frazier, Bowden- (결혼만족도척도의 타당화 연구 -Roach, Frazier, Bowden의 Marital Satisfaction Scal-)

  • 이인수;유영주
    • Journal of Families and Better Life
    • /
    • v.4 no.1
    • /
    • pp.1-13
    • /
    • 1986
  • The purpose of this study was to examine whether the Marital Satisfaction Scale(M.S.S) is properly applicable to Korean couples and further to be devoted to the designed of a marital satisfaction scale for Korea. The scale employed in this study was a Korean adaptation of the M.S.S., which originally included 73 items but was reduced to 48 through a study of its validity by Bowden. This scale was administered to 420 couples. One kindergarten, two elementary schools, one middle school, and four high schools, all of them located in Seoul or its satellite cities were selected for the present study. The results of the study were as follows. 1) Through item analysis, 85.4% of the total items showed a high index of discrimination, i.e, of more than +.40, which all was significant discrimination (P<0.001) excluding two items. 2)Factor analysis, which was oriented toward combining factors proper to theory, showed that the M.S.S. consisted of 6 factors and its total variance was 50.18%. The 6 factors were marital satisfaction (factor 1), marital social psychological companionship (factor 2), obstacles to marriage (factor 3), dissatisfaction of marital relationship (factor 4), attitude of the reasonable marital relationship (factor 5), and cognition of married life (factor 6), 3) internal consistency reliability of the M.S.S. was 0.951 and the reliability of the total items after excluding two through item analysis 0.956 and 0.949 respectively, which showed a high internal consistency 4) The marital satisfaction examined by the M.S.S showed significant differences according to sex, education , income, the duration of marriage, the number of children but no significant differences according to age, and family type. From these results, it will be concluded that the M.S.S is a reliable and valid psychological scale for examining marital satisfaction of Korean couples. The limitations of this study were as follows. 1) It can not be generalized to all Korean couples because the subjects was sampled from Seoul or satellite cities. 2)The factor analysis in this study was oriented toward combining factors proper to theory. However, it was very difficult to find out the factors because the components of the M.S.S. was more or less abstract. Thus, from these results some suggestions follow for developing the marital satisfaction scale proper to Korean couples systematically and scientifically. 1) The subjects must be sampled form urban as well as rural areas 2)Factor analysis in this study is oriented toward finding out factors of the M.S.S. Thus, it is needed to analyze the factor validity or construct validity by multitrait - multimethod of the scale with more theoretical conceptualization. 3) It is necessary to examine whether marital satisfaction shows difference according to couples and psychological variables besides objective demographic variables. 4) More research is needed before the M.S.S. con be adapted into a turely useful marital satisfaction scale for Korea.

  • PDF

Validity of the Happiness-Enhancing Activities and Positive Practices Inventory(HAPPI) Scale in Physical Activities Participation in Korean Old Adults (신체활동참여 한국 노인을 위한 행복증진활동(HAPPI)척도의 구인타당도)

  • Kim, Woo-Kyung
    • Journal of Korea Entertainment Industry Association
    • /
    • v.13 no.7
    • /
    • pp.285-294
    • /
    • 2019
  • The purpose of this study is to assess construct validity and verify the concept of the Happiness-enhancing Activities and Positive Practices Inventory(HAPPI) developed by Henricksen & Stephens(2013), for Korean old adults who participating physical activities with measuring happiness-related propensity. In this study, the research model was confirmed by evidenced based on the content validity, EFA, construct validity of the latent structure analysis with CFA, reliability as internal consistency. Using self-reported questionnaire conducted among 370 participants who physical activities. Total of 344 data were selected. As a result, internal consistency α was acceptable. Evidence-based on convergent and discriminant of the CFA as GFI=.925, CFI .962, TLI .953, and RMSEA .062 appeared significantly. Model goodness-of-fit, C.R. ratio(Critical ratio: estimates/SE) and Squared Multiple Correlations(SMC), and average variance extracted(AVE) was verified with the hypothesis of the model. Therefore, HAPPI validity evidence for the model fit was confirmed. In conclusion, the HAPPI 4 factors and 16 items(Other-focused, Personal recreation and interests, Achievement, Self-Concordant Work, Spiritual and thought-related) has reliable evidence to apply for Korean old adults and applicable assessment of happiness.

Water Quality Assessed by DAIpo and TDI of Bokha Stream and Dal Stream in South-Han River (남한강 지류인 복하천과 달천의 부착규조를 이용한 생물학적 수질평가)

  • Kim, Yong-Jin;Shin, Kyoung-Ae;Lee, Ok-Min
    • Korean Journal of Environmental Biology
    • /
    • v.27 no.4
    • /
    • pp.414-424
    • /
    • 2009
  • The research was to get a biological assessment on Bokha stream and Dal stream, using the physicochemical factors and the epilithic diatom from June, 2008 to March, 2009. As a result, Bokha stream showed high consistency of total phosphorus ($0.041{\sim}0.886\;mg{\cdot}L^{-1}$), and total nitrogen ($0.336{\sim}10.532\;mg{\cdot}L^{-1}$). Also, the concentration of BOD showed most of the sites polluted as $2{\sim}5\;mg\cdot}L^{-1}$ except for the upstream sites. However, excluding downstream sites and other few sites, Dal stream had very clean water quality having total phosphorus less than $0.06\;mg\cdot}L^{-1}$, and total nitrogen less than $2.0\;mg\cdot}L^{-1}$. Out of the 12 total sites, 98 taxa of epilithic diatoms appeared: 67 taxa from Bokha stream, and 78 taxa from Dal stream. The results of the water quality assessment showed that, Bokha stream, having TDI level over 80, was mostly $\alpha$-mesosaprobic, which indicates a severe pollution. On the other hand, Dal stream, having TDI level below 50, was examined to be xenosaprobic and $\alpha$-oligosaprobic, which indicates a clean water quality. The correlation between DAIpo and TDI measured to be high in correlation coefficent (0.83) from the result of correlation analysis. When the water quality is good, there is an inclination of getting high correlation between chemical and biological water quality assessment results. TDI, which uses numerous taxa of diatoms, showed more stability than DAIpo, which uses less amount of taxa of diatoms to get its result.

Assessment of Parallel Computing Performance of Agisoft Metashape for Orthomosaic Generation (정사모자이크 제작을 위한 Agisoft Metashape의 병렬처리 성능 평가)

  • Han, Soohee;Hong, Chang-Ki
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.37 no.6
    • /
    • pp.427-434
    • /
    • 2019
  • In the present study, we assessed the parallel computing performance of Agisoft Metashape for orthomosaic generation, which can implement aerial triangulation, generate a three-dimensional point cloud, and make an orthomosaic based on SfM (Structure from Motion) technology. Due to the nature of SfM, most of the time is spent on Align photos, which runs as a relative orientation, and Build dense cloud, which generates a three-dimensional point cloud. Metashape can parallelize the two processes by using multi-cores of CPU (Central Processing Unit) and GPU (Graphics Processing Unit). An orthomosaic was created from large UAV (Unmanned Aerial Vehicle) images by six conditions combined by three parallel methods (CPU only, GPU only, and CPU + GPU) and two operating systems (Windows and Linux). To assess the consistency of the results of the conditions, RMSE (Root Mean Square Error) of aerial triangulation was measured using ground control points which were automatically detected on the images without human intervention. The results of orthomosaic generation from 521 UAV images of 42.2 million pixels showed that the combination of CPU and GPU showed the best performance using the present system, and Linux showed better performance than Windows in all conditions. However, the RMSE values of aerial triangulation revealed a slight difference within an error range among the combinations. Therefore, Metashape seems to leave things to be desired so that the consistency is obtained regardless of parallel methods and operating systems.

Reliability of Web and Paper-Based Survey Methods for Mibyeong and Cold-Heat Pattern Questionnaire for Korean Medicine Health Assessment: Pilot Study (한의 건강 측정을 위한 미병과 한열설문의 웹과 종이 기반 조사 방법의 신뢰도: 예비연구)

  • Jeong, Kyoungsik;Kim, Hoseok;Lee, Siwoo;Lim, Sueun;Baek, Younghwa
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.8
    • /
    • pp.671-680
    • /
    • 2022
  • This study evaluated the consistency between the web-based and paper-based mibyeong and cold-heat pattern questionnaire, the Korean medicine-based tool for diagnosing and classifying health status. First, a web-based survey was conducted on 72 ordinary people; subsequently, a paper-based survey was conducted after a certain time interval. The equivalence between the web-based and paper-based surveys was evaluated on the basis of the consistency between scores using the Intraclass Correlation Coefficient (ICC) and Bland-Altman methodology. The mibyeong questionnaire showed high reliability for the web-based and paper-based surveys (ICC=0.95, 95% CI 0.92 - 0.97), and the cold-heat pattern questionnaire showed high reliability for both cold syndrome (ICC=0.98, 95% CI 0.96 - 0.99) and heat syndrome (ICC=0.9, 95% CI 0.83 - 0.93). The difference in average scores between the two survey methods was -0.25 for the mibyeong survey, -0.17 for the cold syndrome, and 0.11 for the heat syndrome, showing a similar pattern. Among the respondents, 84% showed positive satisfaction with the web-based survey, and 80% preferred the web-based survey. Overall, this study confirmed the reliability and feasibility of the web-based survey methods for the mibyeong and cold-heat pattern questionnaire. This could be a useful tool for the follow-up of subjects in long-term cohort studies.