• Title/Summary/Keyword: term extraction

Search Result 340, Processing Time 0.026 seconds

Evaluation of English Term Extraction based on Inner/Outer Term Statistics

  • Kang, In-Su
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.4
    • /
    • pp.141-148
    • /
    • 2020
  • Automatic term extraction is to recognize domain-specific terms given a collection of domain-specific text. Previous term extraction methods operate effectively in unsupervised manners which include extracting candidate terms, and assigning importance scores to candidate terms. Regarding the calculation of term importance scores, the study focuses on utilizing sets of inner and outer terms of a candidate term. For a candidate term, its inner terms are shorter terms which belong to the candidate term as components, and its outer terms are longer terms which include the candidate term as their component. This work presents various functions that compute, for a candidate term, term strength from either set of its inner or outer terms. In addition, a scoring method of a term importance is devised based on C-value score and the term strength values obtained from the sets of inner and outer terms. Experimental evaluations using GENIA and ACL RD-TEC 2.0 datasets compare and analyze the effectiveness of the proposed term extraction methods for English. The proposed method performed better than the baseline method by up to 1% and 3% respectively for GENIA and ACL datasets.

Comparison of Significant Term Extraction Based on the Number of Selected Principal Components (주성분 보유수에 따른 중요 용어 추출의 비교)

  • Lee Chang-Beom;Ock Cheol-Young;Park Hyuk-Ro
    • The KIPS Transactions:PartB
    • /
    • v.13B no.3 s.106
    • /
    • pp.329-336
    • /
    • 2006
  • In this paper, we propose a method of significant term extraction within a document. The technique used is Principal Component Analysis(PCA) which is one of the multivariate analysis methods. PCA can sufficiently use term-term relationships within a document by term-term correlations. We use a correlation matrix instead of a covariance matrix between terms for performing PCA. We also try to find out thresholds of both the number of components to be selected and correlation coefficients between selected components and terms. The experimental results on 283 Korean newspaper articles show that the condition of the first six components with correlation coefficients of |0.4| is the best for extracting sentence based on the significant selected terms.

Energy Minimization Based Semantic Video Object Extraction

  • Kim, Dong-Hyun;Choi, Sung-Hwan;Kim, Bong-Joe;Shin, Hyung-Chul;Sohn, Kwang-Hoon
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2010.07a
    • /
    • pp.138-141
    • /
    • 2010
  • In this paper, we propose a semi-automatic method for semantic video object extraction which extracts meaningful objects from an input sequence with one correctly segmented training image. Given one correctly segmented image acquired by the user's interaction in the first frame, the proposed method automatically segments and tracks the objects in the following frames. We formulate the semantic object extraction procedure as an energy minimization problem at the fragment level instead of pixel level. The proposed energy function consists of two terms: data term and smoothness term. The data term is computed by considering patch similarity, color, and motion information. Then, the smoothness term is introduced to enforce the spatial continuity. Finally, iterated conditional modes (ICM) optimization is used to minimize energy function in a globally optimal manner. The proposed semantic video object extraction method provides faithful results for various types of image sequences.

  • PDF

A Study on Keyword Extraction From a Single Document Using Term Clustering (용어 클러스터링을 이용한 단일문서 키워드 추출에 관한 연구)

  • Han, Seung-Hee
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.44 no.3
    • /
    • pp.155-173
    • /
    • 2010
  • In this study, a new keyword extraction algorithm is applied to a single document with term clustering. A single document is divided by multiple passages, and two ways of calculating similarities between two terms are investigated; the first-order similarity and the second-order distributional similarity. In this experiment, the best cluster performance is achieved with a 50-term passage from the second-order distributional similarity. From the results of first experiment, the second-order distribution similarity was also applied to various keyword extraction methods using statistic information of terms. In the second experiment, pf(paragraph frequency) and $tf{\times}ipf$(term frequency by inverse paragraph frequency) were found to improve the overall performance of keyword extraction. Therefore, it showed that the algorithm fulfills the necessary conditions which good keywords should have.

Term Frequency-Inverse Document Frequency (TF-IDF) Technique Using Principal Component Analysis (PCA) with Naive Bayes Classification

  • J.Uma;K.Prabha
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.4
    • /
    • pp.113-118
    • /
    • 2024
  • Pursuance Sentiment Analysis on Twitter is difficult then performance it's used for great review. The present be for the reason to the tweet is extremely small with mostly contain slang, emoticon, and hash tag with other tweet words. A feature extraction stands every technique concerning structure and aspect point beginning particular tweets. The subdivision in a aspect vector is an integer that has a commitment on ascribing a supposition class to a tweet. The cycle of feature extraction is to eradicate the exact quality to get better the accurateness of the classifications models. In this manuscript we proposed Term Frequency-Inverse Document Frequency (TF-IDF) method is to secure Principal Component Analysis (PCA) with Naïve Bayes Classifiers. As the classifications process, the work proposed can produce different aspects from wildly valued feature commencing a Twitter dataset.

An Ontology-based Knowledge Management System - Integrated System of Web Information Extraction and Structuring Knowledge -

  • Mima, Hideki;Matsushima, Katsumori
    • Proceedings of the CALSEC Conference
    • /
    • 2005.03a
    • /
    • pp.55-61
    • /
    • 2005
  • We will introduce a new web-based knowledge management system in progress, in which XML-based web information extraction and our structuring knowledge technologies are combined using ontology-based natural language processing. Our aim is to provide efficient access to heterogeneous information on the web, enabling users to use a wide range of textual and non textual resources, such as newspapers and databases, effortlessly to accelerate knowledge acquisition from such knowledge sources. In order to achieve the efficient knowledge management, we propose at first an XML-based Web information extraction which contains a sophisticated control language to extract data from Web pages. With using standard XML Technologies in the system, our approach can make extracting information easy because of a) detaching rules from processing, b) restricting target for processing, c) Interactive operations for developing extracting rules. Then we propose a structuring knowledge system which includes, 1) automatic term recognition, 2) domain oriented automatic term clustering, 3) similarity-based document retrieval, 4) real-time document clustering, and 5) visualization. The system supports integrating different types of databases (textual and non textual) and retrieving different types of information simultaneously. Through further explanation to the specification and the implementation technique of the system, we will demonstrate how the system can accelerate knowledge acquisition on the Web even for novice users of the field.

  • PDF

Estimation of Future Long-Term Riverbed Fluctuations and Aggregate Extraction Volume Using Climate Change Scenarios: A Case Study of the Nonsan River Basin (기후변화시나리오를 이용한 미래 장기하상변동 및 골재 채취량 산정: 논산천을 사례로)

  • Dae Eop Lee;Min Seok Kim;Hyun Ju Oh
    • Economic and Environmental Geology
    • /
    • v.57 no.2
    • /
    • pp.107-117
    • /
    • 2024
  • The objective of this study is to estimate riverbed fluctuations and the volume of aggregate extraction attributable to climate change. Rainfall-runoff modeling, utilizing the SWAT model based on climate change scenarios, as well as long-term riverbed fluctuation modeling, employing the HEC-RAS model, were conducted for the Nonsan River basin. The analysis of rainfall-runoff and sediment transport under the SSP5-8.5 scenario for the early part of the future indicates that differences in annual precipitation may exceed 600 mm, resulting in a corresponding variation in the basin's sediment discharge by more than 30,000 tons per year. Additionally, long-term riverbed fluctuation modeling of the lower reaches of the Nonsan Stream has identified a potential aggregate extraction area. It is estimated that aggregate extraction could be feasible within a 2.455 km stretch upstream, approximately 4.6 to 6.9 km from the confluence with the Geum River. These findings suggest that the risk of climate crises, such as extreme rainfall or droughts, could increase due to abnormal weather conditions, and the increase in variability could affect long-term aggregate extraction. Therefore, it is considered important to take into account the impact of climate change in future long-term aggregate extraction planning and policy formulation.

FastText and BERT for Automatic Term Extraction (FastText 와 BERT 를 이용한 자동 용어 추출)

  • Choi, Kyu-Hyun;Na, Seung-Hoon
    • Annual Conference on Human and Language Technology
    • /
    • 2021.10a
    • /
    • pp.612-616
    • /
    • 2021
  • 자연어 처리의 다양한 task 들을 잘 수행하기 위해서 텍스트 내에서 적절한 용어를 골라내는 것은 중요하다. 텍스트에서 적절한 용어들을 자동으로 추출하기 위해 다양한 모델들을 학습시켜 용어의 특성을 잘 반영하는 n 그램을 추출할 수 있다. 본 연구에서는 기존에 존재하는 신경망 모델들을 조합하여 자동 용어 추출 성능을 개선할 수 있는 방법들을 제시하고 각각의 결과들을 비교한다.

  • PDF

Cross-Domain Text Sentiment Classification Method Based on the CNN-BiLSTM-TE Model

  • Zeng, Yuyang;Zhang, Ruirui;Yang, Liang;Song, Sujuan
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.818-833
    • /
    • 2021
  • To address the problems of low precision rate, insufficient feature extraction, and poor contextual ability in existing text sentiment analysis methods, a mixed model account of a CNN-BiLSTM-TE (convolutional neural network, bidirectional long short-term memory, and topic extraction) model was proposed. First, Chinese text data was converted into vectors through the method of transfer learning by Word2Vec. Second, local features were extracted by the CNN model. Then, contextual information was extracted by the BiLSTM neural network and the emotional tendency was obtained using softmax. Finally, topics were extracted by the term frequency-inverse document frequency and K-means. Compared with the CNN, BiLSTM, and gate recurrent unit (GRU) models, the CNN-BiLSTM-TE model's F1-score was higher than other models by 0.0147, 0.006, and 0.0052, respectively. Then compared with CNN-LSTM, LSTM-CNN, and BiLSTM-CNN models, the F1-score was higher by 0.0071, 0.0038, and 0.0049, respectively. Experimental results showed that the CNN-BiLSTM-TE model can effectively improve various indicators in application. Lastly, performed scalability verification through a takeaway dataset, which has great value in practical applications.

Semantic Relation Extraction using Pattern Pairs Sharing a Term (용어를 공유하는 패턴 쌍을 이용한 의미 관계 추출)

  • Kim, Se-Jong;Lee, Yong-Hun;Lee, Jong-Hyeok
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.3
    • /
    • pp.221-225
    • /
    • 2009
  • Constructing an ontology using a mass corpus begins with an automatic semantic relation extraction. A general method regards words appearing between terms as patterns which are used to extract semantic relations. However, previous approaches consider only one sentence to extract a pattern, so they cannot extract semantic relations for terms in different sentences. This paper proposes a semantic relation extraction method using pairs of patterns sharing a term, where each pattern is extracted using one of the seed term pair satisfying the target relation. In our experiments, we achieved the accuracy 83.75% improving previous methods by 7.5% in is-${\alpha}$ relation and the accuracy 83.75% improved by 5% in part-of relation. We also present a possibility of improving the recall by the relative recall.