• Title/Summary/Keyword: Document clustering

Search Result 224, Processing Time 0.026 seconds

A Comparative Study on Performance Evaluation of Document Clustering Results (문헌 클러스터링 결과의 성능 평가 방법에 관한 비교 연구)

  • 김정하;이재윤
    • Proceedings of the Korean Society for Information Management Conference
    • /
    • 2000.08a
    • /
    • pp.45-50
    • /
    • 2000
  • 자동분류나 정보검색에 활용되는 문헌 클러스터링 결과의 성능을 평가하는 방법에는 여러가지가 있다. 본 논문에서는 제시된 몇 가지 평가방법의 개념과 특징에 대해 알아본다 학술논문 초록 집합인 KTSET과 신문기사 집합인 KFCM-CL을 대상으로 각각 유사계수를 변화시켜가며 클러스터링한 결과에 대해 각 평가방법을 적응해본 후, 특징과 문제점을 살려 보았다.

  • PDF

Clustering Method Of Plagiarism Document To Use Similarity Syntagma Tree (유사 어절 트리를 이용한 표절 문서의 Clustering 방법)

  • Cheon, Seung-Hwan;Kim, Mi-Young;Lee, Guee-Sang
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11c
    • /
    • pp.2269-2272
    • /
    • 2002
  • 인터넷과 컴퓨터를 이용한 학생들의 과제물을 평가하는데 있어 표절의 용이성으로 인해 정확히 판별하는 것은 매우 어렵고 번거로운 일이다. 특히 동일한 주제에 대해서 작성되는 경우가 많으므로 독자적으로 작성된 문서와 표절되어진 문서를 판별하기가 쉽지 않다. 이것은 클러스터링 하고자 하는 문서들에서 주요 단어들 즉, 색인어들의 출현 빈도를 추출한 뒤 이를 이용하여 가장 적합한 Clustering을 찾는 기존의 정보 검색 방법들과는 전혀 다른 문제이다. 본 논문에서는 과제물의 평가에 지침을 제공할 수 있도록 유사 어절 트리를 이용한 표절 유사도에 따른 Cluster들을 생성하는 방법에 대해 제안한다.

  • PDF

Clustering System Model of Intormation Retrieval using NFC Tag Information (NFC 태그 정보를 이용한 검색 정보의 군집 시스템 모델)

  • Park, Sun;Kim, HyeongGyun;Sim, Su-Jeong
    • Smart Media Journal
    • /
    • v.2 no.3
    • /
    • pp.17-22
    • /
    • 2013
  • The growth of the propagated NFC provides the various services with respect to internet applications, which it can be predicted from the simple internet services to the privated services. This paper proposes the clustering of information retrieval system model using NFC tag of access information for utilizing the similar information of the tag. The proposed model can search the similar information of the tag using the access information of NFC tag. In addition, it can cluster the similar retrieval information into topic cluster for utilizaing users.

  • PDF

Multiview Data Clustering by using Adaptive Spectral Co-clustering (적응형 분광 군집 방법을 이용한 다중 특징 데이터 군집화)

  • Son, Jeong-Woo;Jeon, Junekey;Lee, Sang-Yun;Kim, Sun-Joong
    • Journal of KIISE
    • /
    • v.43 no.6
    • /
    • pp.686-691
    • /
    • 2016
  • In this paper, we introduced the adaptive spectral co-clustering, a spectral clustering for multiview data, especially data with more than three views. In the adaptive spectral co-clustering, the performance is improved by sharing information from diverse views. For the efficiency in information sharing, a co-training approach is adopted. In the co-training step, a set of parameters are estimated to make all views in data maximally independent, and then, information is shared with respect to estimated parameters. This co-training step increases the efficiency of information sharing comparing with ordinary feature concatenation and co-training methods that assume the independence among views. The adaptive spectral co-clustering was evaluated with synthetic dataset and multi lingual document dataset. The experimental results indicated the efficiency of the adaptive spectral co-clustering with the performances in every iterations and similarity matrix generated with information sharing.

Clustering Meta Information of K-Pop Girl Groups Using Term Frequency-inverse Document Frequency Vectorization (단어-역문서 빈도 벡터화를 통한 한국 걸그룹의 음반 메타 정보 군집화)

  • JoonSeo Hyeon;JaeHyuk Cho
    • Journal of Platform Technology
    • /
    • v.11 no.3
    • /
    • pp.12-23
    • /
    • 2023
  • In the 2020s, the K-Pop market has been dominated by girl groups over boy groups and the fourth generation over the third generation. This paper presents methods and results on lyric clustering to investigate whether the generation of girl groups has started to change. We collected meta-information data for 1469 songs of 47 groups released from 2013 to 2022 and classified them into lyric information and non-lyric meta-information and quantified them respectively. The lyrics information was preprocessed by applying word-translation frequency vectorization based on previous studies and then selecting only the top vector values. Non-lyric meta-information was preprocessed and applied with One-Hot Encoding to reduce the bias of using only lyric information and show better clustering results. The clustering performance on the preprocessed data is 129%, 45% higher for Spherical K-Means' Silhouette Score and Calinski-Harabasz Score, respectively, compared to Hierarchical Clustering. This paper is expected to contribute to the study of Korean popular song development and girl group lyrics analysis and clustering.

  • PDF

An Automatic Classification System of Korean Documents Using Weight for Keywords of Document and Word Cluster (문서의 주제어별 가중치 부여와 단어 군집을 이용한 한국어 문서 자동 분류 시스템)

  • Hur, Jun-Hui;Choi, Jun-Hyeog;Lee, Jung-Hyun;Kim, Joong-Bae;Rim, Kee-Wook
    • The KIPS Transactions:PartB
    • /
    • v.8B no.5
    • /
    • pp.447-454
    • /
    • 2001
  • The automatic document classification is a method that assigns unlabeled documents to the existing classes. The automatic document classification can be applied to a classification of news group articles, a classification of web documents, showing more precise results of Information Retrieval using a learning of users. In this paper, we use the weighted Bayesian classifier that weights with keywords of a document to improve the classification accuracy. If the system cant classify a document properly because of the lack of the number of words as the feature of a document, it uses relevance word cluster to supplement the feature of a document. The clusters are made by the automatic word clustering from the corpus. As the result, the proposed system outperformed existing classification system in the classification accuracy on Korean documents.

  • PDF

High-Speed Self-Organzing Map for Document Clustering

  • Rojanavasu, Ponthap;Pinngern, Ouen
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2003.10a
    • /
    • pp.1056-1059
    • /
    • 2003
  • Self-Oranizing Map(SOM) is an unsupervised neural network providing cluster analysis of high dimensional input data. The output from the SOM is represented in map that help us to explore data. The weak point of conventional SOM is when the map is large, it take a long time to train the data. The computing time is known to be O(MN) for trainning to find the winning node (M,N are the number of nodes in width and height of the map). This paper presents a new method to reduce the computing time by creating new map. Each node in a new map is the centroid of nodes' group that are in the original map. After create a new map, we find the winning node of this map, then find the winning node in original map only in nodes that are represented by the winning node from the new map. This new method is called "High Speed Self-Oranizing Map"(HS-SOM). Our experiment use HS-SOM to cluster documents and compare with SOM. The results from the experiment shows that HS-SOM can reduce computing time by 30%-50% over conventional SOM.

  • PDF

XML Clustering Technique by Genetic Algorithm (유전자 알고리즘을 통한 XML 군집화 방법)

  • Kim, Woo-Saeng
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.49 no.3
    • /
    • pp.1-7
    • /
    • 2012
  • Recently, researches are studied in developing efficient techniques for accessing, querying, and managing XML documents which are frequently used in the Internet. In this paper, we propose a new method to cluster XML documents efficiently. An element of a XML document corresponds to a node of the corresponding tree and an inclusion relationship of the document corresponds to a relationship between parent and child node of the tree. Therefore, similar XML documents are similar to the node's name and level of the corresponding trees. We make evaluation function with this characteristic to cluster XML documents by genetic algorithm. The experiment shows that our proposed method has better performance than other existing methods.

An Ontology-based Knowledge Management System - Integrated System of Web Information Extraction and Structuring Knowledge -

  • Mima, Hideki;Matsushima, Katsumori
    • Proceedings of the CALSEC Conference
    • /
    • 2005.03a
    • /
    • pp.55-61
    • /
    • 2005
  • We will introduce a new web-based knowledge management system in progress, in which XML-based web information extraction and our structuring knowledge technologies are combined using ontology-based natural language processing. Our aim is to provide efficient access to heterogeneous information on the web, enabling users to use a wide range of textual and non textual resources, such as newspapers and databases, effortlessly to accelerate knowledge acquisition from such knowledge sources. In order to achieve the efficient knowledge management, we propose at first an XML-based Web information extraction which contains a sophisticated control language to extract data from Web pages. With using standard XML Technologies in the system, our approach can make extracting information easy because of a) detaching rules from processing, b) restricting target for processing, c) Interactive operations for developing extracting rules. Then we propose a structuring knowledge system which includes, 1) automatic term recognition, 2) domain oriented automatic term clustering, 3) similarity-based document retrieval, 4) real-time document clustering, and 5) visualization. The system supports integrating different types of databases (textual and non textual) and retrieving different types of information simultaneously. Through further explanation to the specification and the implementation technique of the system, we will demonstrate how the system can accelerate knowledge acquisition on the Web even for novice users of the field.

  • PDF

Performance Improvement by Cluster Analysis in Korean-English and Japanese-English Cross-Language Information Retrieval (한국어-영어/일본어-영어 교차언어정보검색에서 클러스터 분석을 통한 성능 향상)

  • Lee, Kyung-Soon
    • The KIPS Transactions:PartB
    • /
    • v.11B no.2
    • /
    • pp.233-240
    • /
    • 2004
  • This paper presents a method to implicitly resolve ambiguities using dynamic incremental clustering in Korean-to-English and Japanese-to-English cross-language information retrieval (CLIR). The main objective of this paper shows that document clusters can effectively resolve the ambiguities tremendously increased in translated queries as well as take into account the context of all the terms in a document. In the framework we propose, a query in Korean/Japanese is first translated into English by looking up bilingual dictionaries, then documents are retrieved for the translated query terms based on the vector space retrieval model or the probabilistic retrieval model. For the top-ranked retrieved documents, query-oriented document clusters are incrementally created and the weight of each retrieved document is re-calculated by using the clusters. In the experiment based on TREC test collection, our method achieved 39.41% and 36.79% improvement for translated queries without ambiguity resolution in Korean-to-English CLIR, and 17.89% and 30.46% improvements in Japanese-to-English CLIR, on the vector space retrieval and on the probabilistic retrieval, respectively. Our method achieved 12.30% improvements for all translation queries, compared with blind feedback in Korean-to-English CLIR. These results indicate that cluster analysis help to resolve ambiguity.