• Annual Conference on Human and Language Technology
• /
• 2006.10e
• /
• pp.98-106
• /
• 2006

문서분류기의 개발에 있어 교사학습기법을 이용할 경우 많은 양의 사람에 의한 범주 부착 말뭉치가 필요하다. 그러나 이의 구축은 많은 시간과 노력을 필요로 한다. 최근 이러한 범주 부착 말뭉치 대신 원시말뭉치와 범주마다 약간의 씨앗 정보를 이용하여 학습을 수행하여 문서분류기를 개발하는 방법론이 제시되었다. 본 논문에서는 이 방법론 하에서 다른 연구에서의 결과보다 좋은 성능을 나타내는 비교사 학습 기법을 소개한다. 본 논문에서 제시하는 기법의 특징은 씨앗 단어에서 출발하여 평균상호정보를 이용하여 다른 대표단어 및 그들의 가중치를 학습한 다음, 정보검색에서 많이 사용하는 기술을 이용하여 그 가중치를 갱신하는 것이다. 그리고 이 과정을 반복 수행하여 최종적으로 높은 성능의 시스템을 개발할 수 있음을 제시하였다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2003.05a
• /
• pp.261-264
• /
• 2003

본 연구는 바이그램을 이용하여 자동문서범주화 성능을 향상시키는 알고리즘의 개발을 목표로 한다. 기존의 문서 범주화 알고리즘의 장단점을 비교하여 개선된 바이그램 추출 알고리즘을 구현하고, 이 알고리즘을 실험한 결과 Reuters-21579 data set은 개별 단어를 사용하여 시험한 결과보다 단어+바이그램을 사용하였을 경우 BEP은 2.07%, F1은 1.40% 향상률을 보였고, Korea-web data set은 BEP의 8.12%, F1의 6.25% 향상을 보였다. 이와 같은 실험결과는 단어를 사용한 경우보다 단어+바이그램을 사용한 자동문서 범주화 시스템이 더 효율적이라는 것을 보여준다.

• International Journal of Contents
• /
• v.1 no.2
• /
• pp.26-30
• /
• 2005

According to the fast growth of information on the Internet, it is becoming increasingly difficult to find and organize useful information. To reduce information overload, it needs to exploit automatic text classification for handling enormous documents. Support Vector Machine (SVM) is a model that is calculated as a weighted sum of kernel function outputs. This paper describes a document classifier for web documents in the fields of Information Technology and uses SVM to learn a model, which is constructed from the training sets and its representative terms. The basic idea is to exploit the representative terms meaning distribution in coherent thematic texts of each category by simple statistics methods. Vector-space model is applied to represent documents in the categories by using feature selection scheme based on TFiDF. We apply a category factor which represents effects in category of any term to the feature selection. Experiments show the results of categorization and the correlation of vector length.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2010.10a
• /
• pp.695-697
• /
• 2010

문서에 포함되는 어휘는 문서 분류의 정보를 가지므로 문서를 분석하여 유용한 단어를 추출하는 것은 다양한 서비스와 연계되어 사용될 수 있어 매우 유용한 일이다. 문서 자동 분류에서는 분류자질 선정 방식에 따라 분류정확도가 서로 달라질 수 있으며, 문서에서 추출되는 유용한 단어에 따라 인지되는 분야가 달라질 수 있다. 이에 본 논문에서는 각 문서에 포함되는 단어에 대한 카이제곱 통계량 점수를 사용하여 단어별 문서 분류에 대한 단어의 자질을 평가하고 문서의 분류별 유용한 단어를 자동 추출하는 방법을 제안하고 개발한다.

• Proceedings of the Korean Society for Information Management Conference
• /
• 2014.08a
• /
• pp.127-130
• /
• 2014

이 연구에서는 도서관의 주요 장서에 해당하는 단행본 도서에 대한 자동 분류를 적용가능한지 알아보고자 하였다. 분류자질로 메타데이터인 서명, 목차, 책 소개를 사용하였으며, 다양한 자질 가중치를 적용하여 581건의 단행본 도서를 통해 kNN 분류기의 분류성능을 파악하였다. 실험 결과 이들 메타데이터를 모두 사용하였을 때 가장 좋은 분류성능을 가져왔으며, 실험문헌집단의 규모가 작은 한계가 있지만 로그 TF를 취한 가중치 방법이 좋은 성능을 가져왔다.

• Journal of KIISE:Software and Applications
• /
• v.34 no.2
• /
• pp.160-168
• /
• 2007

For developing a text classifier using supervised learning, a manually labeled corpus of large size is required. However, it takes a lot of time and human effort. Recently a research paradigm was proposed to use a raw corpus and a small amount of seed information instead of manually labeled corpus. In this paper we introduce an unsupervised learning method that makes it possible to achieve better performance than other related works. The characteristics of our approach is that average mutual information is used to learn representative words and their weights and then update of the weights is done using a technique inspired by the works in information retrieval. By iterating this teaming process it was shown that a high performance system can be developed.

• Journal of the Korean Society of Clothing and Textiles
• /
• v.40 no.6
• /
• pp.1034-1044
• /
• 2016

This study seeks a method to objectively evaluate sensibility based on Big Data in the field of design. In order to do so, this study examined the sensibility responses on design factors for the public through a network analysis of texts displayed in social media. 'Hanbok', a formal clothing that represents Korea, was selected as the subject for the research methodology. We then collected 47,677 keywords related to Hanbok from 12,000 posts on Naver blogs from January $1^{st}$ to December $31^{st}$ 2015 and that analyzed using social matrix (a Big Data analysis software) rather than using previous survey methods. We also derived 56 key-words related to design elements and sensibility responses of Hanbok. Centrality analysis and CONCOR analysis were conducted using Ucinet6. The visualization of the network text analysis allowed the categorization of the main design factors of Hanbok with evaluation terms that mean positive, negative, and neutral sensibility responses. We also derived key evaluation factors for Hanbok as fitting, rationality, trend, and uniqueness. The evaluation terms extracted based on natural language processing technologies of atypical data have validity as a scale for evaluation and are expected to be suitable for utilization in an index for sensibility evaluation that supplements the limits of previous surveys and statistical analysis methods. The network text analysis method used in this study provides new guidelines for the use of Big Data involving sensibility evaluation methods in the field of design.

• Journal of the Korean Society of Food Culture
• /
• v.38 no.1
• /
• pp.1-14
• /
• 2023

This study was conducted to explore the change in the market issues on HMR (Home Meal Replacements) using local foods after the COVID-19 outbreak. Online text data were collected from internet news, social media posts, and web documents before (from January 2016 to December 2019) and after (from January 2020 to November 2022) the COVID-19 outbreak. TF-IDF analysis showed that 'Trend', 'Market', 'Consumption', and 'Food service industry' were the major keywords before the COVID-19 outbreak, whereas 'Wanju-gun', 'Distribution', 'Development', and 'Meal-kit' were main keywords after the COVID-19 outbreak. The results of topic modeling analysis and categorization showed that after the COVID-19 outbreak, the 'Market' category included 'Non-face-to-face market' instead of 'Event,' and 'Delivery' instead of 'Distribution'. In the 'Product' category, 'Marketing' was included instead of 'Trend'. Additionally, in the 'Support' category, 'Start-up' and 'School food service' appeared as new topics after the COVID-19 outbreak. In conclusion, this study showed that meaningful change had occurred in market issues on HMR using local foods after the COVID-19 outbreak. Therefore, governments should take advantage of such market opportunity by implementing policy and programs to promote the development and marketing of HMR using local foods.

• Proceedings of the Korea Contents Association Conference
• /
• 2007.11a
• /
• pp.435-439
• /
• 2007

Document categorization system is important in the internet age in which huge number of documents are created and need to be dealt with. By this reason a lot of research has been done in this field. For the development of the system, a supervised learning method is widely used. This approach needs a test collection as a prerequisite. For the case of English, several test collections are available which provide a lot of help for developing systems and doing research. But no public test collections have been reported and are not available in the case of Korean. To improve the situation for Korean we are undergoing the construction of a Korean test collection. In this paper the approaches being used and current stage of the collection will be described.

• Journal of the Korean Society for information Management
• /
• v.40 no.1
• /
• pp.1-21
• /
• 2023

This study specifically reviewed the performance of the ranking schemes as an efficient feature selection method for text classification. Until now, feature ranking schemes are mostly based on document frequency, and relatively few cases have used the term frequency. Therefore, the performance of single ranking metrics using term frequency and document frequency individually was examined as a feature selection method for text classification, and then the performance of combination ranking schemes using both was reviewed. Specifically, a classification experiment was conducted in an environment using two data sets (Reuters-21578, 20NG) and five classifiers (SVM, NB, ROC, TRA, RNN), and to secure the reliability of the results, 5-Fold cross-validation and t-test were applied. As a result, as a single ranking scheme, the document frequency-based single ranking metric (chi) showed good performance overall. In addition, it was found that there was no significant difference between the highest-performance single ranking and the combination ranking schemes. Therefore, in an environment where sufficient learning documents can be secured in text classification, it is more efficient to use a single ranking metric (chi) based on document frequency as a feature selection method.