• Title/Summary/Keyword: Document clustering

Search Result 224, Processing Time 0.03 seconds

Development of A Web Mining System Based On Document Similarity (문서 유사도 기반의 웹 마이닝 시스템 개발)

  • 이강찬;민재홍;박기식;임동순;우훈식
    • The Journal of Society for e-Business Studies
    • /
    • v.7 no.1
    • /
    • pp.75-86
    • /
    • 2002
  • In this study, we proposed design issues and structure of a web mining system and develop a system for the purpose of knowledge integration under world wide web environments resulted from our developing experiences. The developed system consists of three main functions: 1) gathering documents utilizing a search agent; 2) determining similarity coefficients between any two documents from term frequencies; 3) clustering documents based on similarity coefficients. It is believed that the developed system can be utilized for discovery of knowledge in relatively narrow domains such as news classification, index term generation in knowledge management.

  • PDF

Trends in Genomics & Informatics: a statistical review of publications from 2003 to 2018 focusing on the most-studied genes and document clusters

  • Kim, Ji-Hyeon;Nam, Hee-Jo;Park, Hyun-Seok
    • Genomics & Informatics
    • /
    • v.17 no.3
    • /
    • pp.25.1-25.6
    • /
    • 2019
  • Genomics & Informatics (NLM title abbreviation: Genomics Inform) is the official journal of the Korea Genome Organization. Herein, we conduct a statistical analysis of the publications of Genomics & Informatics over the 16 years since its inception, with a particular focus on issues relating to article categories, word clouds, and the most-studied genes, drawing on recent reviews of the use of word frequencies in journal articles. Trends in the studies published in Genomics & Informatics are discussed both individually and collectively.

Document clustering based on summarized document using K-means algorithm (요약 문서 기반 문서 클러스터링)

  • Oh, Hyung-Jin;Ko, Ji-Hyun;An, Dong-Un;Chung, Sung-Jong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.04a
    • /
    • pp.589-592
    • /
    • 2002
  • 정보검색 시스템에서 문서 클러스터링 기법은 사용자 질의에 대하여 검색된 문서를 문서간의 관련도에 따라 클러스터로 구성하고 사용자에게 검색 결과로 보여주는 것이다. 본 논문에서는 사용자의 질의에 대하여 검색된 문서를 자동 문서 요약기를 통해 얻은 요약 문서와 문서 전문을 문서들간의 유사도를 기반으로 동적으로 클러스터링 한다. 구현한 시스템의 클러스터링 효과를 검증한 결과 검색된 문서 전문을 클러스터링 한 방식에 비해 요약 문서를 클러스터링 한 방식이 정확률 측면에서 더 나은 성능을 보였다.

  • PDF

Document Embedding and Image Content Analysis for Improving News Clustering System (뉴스 클러스터링 개선을 위한 문서 임베딩 및 이미지 분석 자질의 활용)

  • Kim, Siyeon;Kim, Sang-Bum
    • Annual Conference on Human and Language Technology
    • /
    • 2015.10a
    • /
    • pp.104-108
    • /
    • 2015
  • 많은 양의 뉴스가 생성됨에 따라 이를 효과적으로 정리하는 기법이 최근 활발히 연구되어왔다. 그 중 뉴스클러스터링은 두 뉴스가 동일사건을 다루는지를 판정하는 분류기의 성능에 의존적인데, 대부분의 경우 BoW(Bag-of-Words)기반 벡터유사도를 사용하고 있다. 본 논문에서는 BoW기반의 벡터유사도 뿐 아니라 두 문서에 포함된 사진들의 유사성 및 주제의 관련성을 측정, 이를 분류기의 자질로 추가하여 두 뉴스가 동일사건을 다루는지 판정하는 분류기의 성능을 개선하는 방법을 제안한다. 사진들의 유사성 및 주제의 관련성은 최근 각광을 받는 딥러닝기반 CNN과 신경망기반 문서임베딩을 통해 측정하였다. 실험결과 기존의 BoW기반 벡터유사도에 의한 분류기의 성능에 비해 제안하는 두 자질을 사용하였을 경우 3.4%의 성능 향상을 보여주었다.

  • PDF

A Two-Stage Document Page Segmentation Method using Morphological Distance Map and RBF Network (거리 사상 함수 및 RBF 네트워크의 2단계 알고리즘을 적용한 서류 레이아웃 분할 방법)

  • Shin, Hyun-Kyung
    • Journal of KIISE:Software and Applications
    • /
    • v.35 no.9
    • /
    • pp.547-553
    • /
    • 2008
  • We propose a two-stage document layout segmentation method. At the first stage, as top-down segmentation, morphological distance map algorithm extracts a collection of rectangular regions from a given input image. This preliminary result from the first stage is employed as input parameters for the process of next stage. At the second stage, a machine-learning algorithm is adopted RBF network, one of neural networks based on statistical model, is selected. In order for constructing the hidden layer of RBF network, a data clustering technique bared on the self-organizing property of Kohonen network is utilized. We present a result showing that the supervised neural network, trained by 300 number of sample data, improves the preliminary results of the first stage.

Web Document Clustering for Specific Subject Information Using WordNet and HTML Tags (WordNet과 HTML 태그를 활용한 특정영역 정보의 웹 문서 분류)

  • 조은휘;변영태
    • Proceedings of the Korean Society for Cognitive Science Conference
    • /
    • 2002.05a
    • /
    • pp.28-32
    • /
    • 2002
  • 웹 상의 많은 정보들 속에서 사용자가 원하는 정보를 찾아내는 일은 쉽지 않다. 사용자가 의도하는 양질의 정보 제공을 위해 특정 영역과 관련한 정보 제공 시스템이 .개발되고 있다. 이전 시스템은 특정 영역 관련 지식베이스를 토대로 하여 웹 문서를 수집해 놓고, 사용자에게 정보를 제공한다. 본 논문에서는 전문 사이트 내에 문서간의 유사성을 토대로 하여 동물 영역에 대한 효과적인 문서 클러스타링(clustering)에 관해 실험하였다. 기존의 방법에서는 문서의 분류나 질의어와 관련한 문서 선택이나 순위 결정이 주로 텀(term)을 바탕으로 하고 있다. 본 논문에서는 각 문서 내의 텀 뿐만 아니라 HTML 태그(tag), 지식베이스에 WordNet의 계층구조를 적용한 data를 활용하고, SVD(Singular Value Decomposition)를 사용하여 문서간의 관계를 밝혀내어 문서 분류 및 수집에 이용하였다. 특정 영역의 전문 문서를 많이 제공하는 사이트에 적용하여 좋은 결과를 볼 수 있었다.

  • PDF

A Dynamic Ontology-based Multi-Agent Context-Awareness User Profile Construction Method for Personalized Information Retrieval

  • Gao, Qian;Cho, Young Im
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.12 no.4
    • /
    • pp.270-276
    • /
    • 2012
  • With the increase in amount of data and information available on the web, there have been high demands on personalized information retrieval services to provide context-aware services for the web users. This paper proposes a novel dynamic multi-agent context-awareness user profile construction method based on ontology to incorporate concepts and properties to model the user profile. This method comprehensively considers the frequency and the specific of the concept in one document and its corresponding domain ontology to construct the user profile, based on which, a fuzzy c-means clustering method is adopted to cluster the user's interest domain, and a dynamic update policy is adopted to continuously consider the change of the users' interest. The simulation result shows that along with the gradual perfection of the our user profile, our proposed system is better than traditional semantic based retrieval system in terms of the Recall Ratio and Precision Ratio.

XML Document Clustering Technique by K-means algorithm through PCA (주성분 분석의 K 평균 알고리즘을 통한 XML 문서 군집화 기법)

  • Kim, Woo-Saeng
    • The KIPS Transactions:PartD
    • /
    • v.18D no.5
    • /
    • pp.339-342
    • /
    • 2011
  • Recently, researches are studied in developing efficient techniques for accessing, querying, and storing XML documents which are frequently used in the Internet. In this paper, we propose a new method to cluster XML documents efficiently. We use a K-means algorithm with a Principal Component Analysis(PCA) to cluster XML documents after they are represented by vectors in the feature vector space by transferring them as names and levels of the elements of the corresponding trees. The experiment shows that our proposed method has a good result.

Estimation of the journal distance of Genomics & Informatics from other bioinformatics-driven journals, 2003-2018

  • Oh, Ji-Hye;Nam, Hee-Jo;Park, Hyun-Seok
    • Genomics & Informatics
    • /
    • v.19 no.4
    • /
    • pp.51.1-51.8
    • /
    • 2021
  • This study explored the trends of Genomics & Informatics during the period of 2003-2018 in comparison with 11 other scholarly journals: BMC Bioinformatics, Algorithms for Molecular Biology: AMB, BMC Systems Biology, Journal of Computational Biology, Briefings in Bioinformatics, BMC Genomics, Nucleic Acids Research, American Journal of Human Genetics, Oncogenesis, Disease Markers, and Microarrays. In total, 22,423 research articles were reviewed. Content analysis was the main method employed in the current research. The results were interpreted using descriptive analysis, a clustering analysis, word embedding, and deep learning techniques. Trends are discussed for the 12 journals, both individually and collectively. This is an extension of our previous study (PMCID: PMC6808643).

KOREAN TOPIC MODELING USING MATRIX DECOMPOSITION

  • June-Ho Lee;Hyun-Min Kim
    • East Asian mathematical journal
    • /
    • v.40 no.3
    • /
    • pp.307-318
    • /
    • 2024
  • This paper explores the application of matrix factorization, specifically CUR decomposition, in the clustering of Korean language documents by topic. It addresses the unique challenges of Natural Language Processing (NLP) in dealing with the Korean language's distinctive features, such as agglutinative words and morphological ambiguity. The study compares the effectiveness of Latent Semantic Analysis (LSA) using CUR decomposition with the classical Singular Value Decomposition (SVD) method in the context of Korean text. Experiments are conducted using Korean Wikipedia documents and newspaper data, providing insight into the accuracy and efficiency of these techniques. The findings demonstrate the potential of CUR decomposition to improve the accuracy of document clustering in Korean, offering a valuable approach to text mining and information retrieval in agglutinative languages.