• Title/Summary/Keyword: word extraction

Search Result 233, Processing Time 0.024 seconds

Text extraction from camera based document image (카메라 기반 문서영상에서의 문자 추출)

  • 박희주;김진호
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.8 no.2
    • /
    • pp.14-20
    • /
    • 2003
  • This paper presents a text extraction method of camera based document image. It is more difficult to recognize camera based document image in comparison with scanner based image because of segmentation problem due to variable lighting condition and versatile fonts. Both document binarization and character extraction are important processes to recognize camera based document image. After converting color image into grey level image, gray level normalization is used to extract character region independent of lighting condition and background image. Local adaptive binarization method is then used to extract character from the background after the removal of noise. In this character extraction step, the information of the horizontal and vertical projection and the connected components is used to extract character line, word region and character region. To evaluate the proposed method, we have experimented with documents mixed Hangul, English, symbols and digits of the ETRI database. An encouraging binarization and character extraction results have been obtained.

  • PDF

An Algorithm for Referential Integrity Relations Extraction using Similarity Comparison of RDB (유사성 비교를 통한 RDB의 참조 무결성 관계 추출 알고리즘)

  • Kim, Jang-Won;Jeong, Dong-Won;Kim, Jin-Hyung;Baik, Doo-Kwon
    • Journal of the Korea Society for Simulation
    • /
    • v.15 no.3
    • /
    • pp.115-124
    • /
    • 2006
  • XML is rapidly becoming technologies for information exchange and representation. It causes many research issues such as semantic modeling methods, security, conversion far interoperability with other models, and so on. Especially, the most important issue for its practical application is how to achieve the interoperability between XML model and relational model. Until now, many suggestions have been proposed to achieve it. However several problems still remain. Most of all, the exiting methods do not consider implicit referential integrity relations, and it causes incorrect data delivery. One method to do this has been proposed with the restriction where one semantic is defined as only one same name in a given database. In real database world, this restriction cannot provide the application and extensibility. This paper proposes a noble conversion (RDB-to-XML) algorithm based on the similarity checking technique. The key point of our method is how to find implicit referential integrity relations between different field names presenting one same semantic. To resolve it, we define an enhanced implicity referentiai integrity relations extraction algorithm based on a widely used ontology, WordNet. The proposed conversion algorithm is more practical than the previous-similar approach.

  • PDF

Ontology - Based Intelligent Rule Components Extraction (온톨로지 기반 지능형 규칙 구성요소 추출에 관한 연구)

  • Kim U-Ju;Chae Sang-Yong;Park Sang-Eon
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2006.06a
    • /
    • pp.237-244
    • /
    • 2006
  • 시맨틱 웹 관련연구가 증가함에 따라 하나의 관련분야로 규칙기반 시스템 동의 지능적인 웹 환경에 대한 기대 역시 커지고 있다. 하지만 규칙기반 시스템을 활용하기에는 아직도 규칙습득이 많은 제약이 되고 있다. 규칙습득은 웹으로부터 필요한 규칙을 습득하는 일련의 방법인데, 이러한 규칙을 습득하기 위해서는 규칙구성요소를 먼저 식별해야만 한다. 그러나 이러한 규칙을 식별하는 작업은 대부분 지식관리자의 수작업에 의해 이루어지고 있다. 본 연구의 목적은 웹으로부터 규칙구성요소 식별을 최대한 자동화하고 지식관리자의 수작업을 최소화함으로써 그 부담을 줄여 주는 데 있다. 이러한 방법으로는 온톨로지를 근간으로 하여 웹 페이지와의 문자열 비교, 이러한 비교의 한계를 극복하기 위한 확장등의 방법이 있다. 첫 번째 방법은 온툴로지 기반으로 규칙식별 할 웹 페이지와 비교를 통해 지식관리자의 규칙식별 과정을 최대한 자동화하여 주는 것이다. 여기서 만약 현재 규칙을 식별하고자 하는 웹 사이트와 유사한 시스템의 규칙들을 활용하여 일반화 된 온툴로지가 구축되었다면, 이 온톨로지를 기반으로 규칙을 식별하고자 하는 웹사이트와의 비교를 통해 규칙구성요소를 자동화하여 추출 할 수 있다. 이러한 온툴로지를 기반으로 규칙을 식별하기 위해서는 문자열 비교 기법을 사용하게 된다. 하지만 단순한 문자열 비교 기법만으로는 규칙을 식별하는 데에 자연어 처리에 대한 한계가 있다. 이를 극복하기 위해 다음의 두 번째 방법을 사용하고자 한다. 두 번째 방법은 정형화되지 않은 정보들을 확장하여 사용하는 것이다. 우선 찾고자 하는 단어들의 원형을 찾기 위한 스테밍 알고리즘 기법, WordNet을 이용하여 동의어 유의어등으로 확장을 하는 WordNet Expansion 기법, 의미 유사도를 측정하기 위한 방법인 Semantic Similarity Measure 등을 단계적으로 수행하여 자동화되고 정확한 규칙식별을 하고자 한다. 이러한 방법들의 조합으로 인하여 규칙구성요소 추출이 되지 않을 후보 단어들의 수를 줄여서 보다 더 정확하고, 지능적인 규칙구성요소 추출 방법론을 제시하고 구현하여 지식관리자의 규칙습득에 대한 부담을 줄여 주고자 한다.

  • PDF

A Korean Emotion Features Extraction Method and Their Availability Evaluation for Sentiment Classification (감정 분류를 위한 한국어 감정 자질 추출 기법과 감정 자질의 유용성 평가)

  • Hwang, Jae-Won;Ko, Young-Joong
    • Korean Journal of Cognitive Science
    • /
    • v.19 no.4
    • /
    • pp.499-517
    • /
    • 2008
  • In this paper, we propose an effective emotion feature extraction method for Korean and evaluate their availability in sentiment classification. Korean emotion features are expanded from several representative emotion words and they play an important role in building in an effective sentiment classification system. Firstly, synonym information of English word thesaurus is used to extract effective emotion features and then the extracted English emotion features are translated into Korean. To evaluate the extracted Korean emotion features, we represent each document using the extracted features and classify it using SVM(Support Vector Machine). In experimental results, the sentiment classification system using the extracted Korean emotion features obtained more improved performance(14.1%) than the system using content-words based features which have generally used in common text classification systems.

  • PDF

Feature Extraction of Radar Signals Using Streaming Process (스트리밍 처리에 의한 레이더 신호 특성 추출)

  • Kim, Gwan-Tae;Ju, Young-Kwan;Jeon, Joongnam
    • Journal of Convergence for Information Technology
    • /
    • v.10 no.12
    • /
    • pp.31-38
    • /
    • 2020
  • Radar signal identification of electronic warfare is a technology that recognizes the pulse repetition interval (PRI) from a set of pulse description words (PDWs) generated by the signal receiver. Conventionally batch processing is widely used in which a number of PDWs are collected as a unit and identifies PRI from the batch. In this paper, we propose a feature extraction algorithm based on the streaming process. This technique does not wait to form a batch. Whenever a PDW(Pulse Description Word) is generated from the signal receiver, the streaming process tries to form a cluster of PDWs, and makes the DTOA (Difference of Time of Arrival) histogram, finds out the frame PRI based on the concentration ratio, and decides the number of stagger stages. Experiments proved that the proposed algorithm derives stable recognition results as the cluster size increases.

A Study on Extracting the Document Text for Unallocated Areas of Data Fragments (비할당 영역 데이터 파편의 문서 텍스트 추출 방안에 관한 연구)

  • Yoo, Byeong-Yeong;Park, Jung-Heum;Bang, Je-Wan;Lee, Sang-Jin
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.20 no.6
    • /
    • pp.43-51
    • /
    • 2010
  • It is meaningful to investigate data in unallocated space because we can investigate the deleted data. Consecutively complete file recovery using the File Carving is possible in unallocated area, but noncontiguous or incomplete data recovery is impossible. Typically, the analysis of the data fragments are needed because they should contain large amounts of information. Microsoft Word, Excel, PowerPoint and PDF document file's text are stored using compression or specific document format. If the part of aforementioned document file was stored in unallocated data fragment, text extraction is possible using specific document format. In this paper, we suggest the method of extracting a particular document file text in unallocated data fragment.

A Study on the Effectiveness of Using Keywords in Book Reviews for Customized Book Recommendation for Each Personality Type (성격유형별 선호도서 추천을 위한 서평 키워드 활용의 유효성 연구)

  • Cha, Yeon-Hee;Choi, Sung-Pil
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.55 no.3
    • /
    • pp.343-372
    • /
    • 2021
  • The purpose of this study is to select keywords that can recommend books by personality type, and to test whether the chosen keywords can be actually used in the categorization and customized recommendation of books for each personality type. To achieve the research goal, this study chose books that match the level of fifth and sixth grade elementary school students and first grade middle school students and commissioned an expert group to categorize the books into groups that are preferred by each personality type. As a result of the classification, half of the books in which more than five experts agreed showed high consensus. In addition, the results of classifying books by personality type with keywords extracted by the automatic word extraction system by collecting the book review data of the selected books were similar to the results of the final judgement by the expert group, except for a few books. In conclusion, this study proved that it is possible to classify and recommend books that are likely to be preferred by different personality types using review keywords.

Automatic Extraction of Opinion Words from Korean Product Reviews Using the k-Structure (k-Structure를 이용한 한국어 상품평 단어 자동 추출 방법)

  • Kang, Han-Hoon;Yoo, Seong-Joon;Han, Dong-Il
    • Journal of KIISE:Software and Applications
    • /
    • v.37 no.6
    • /
    • pp.470-479
    • /
    • 2010
  • In relation to the extraction of opinion words, it may be difficult to directly apply most of the methods suggested in existing English studies to the Korean language. Additionally, the manual method suggested by studies in Korea poses a problem with the extraction of opinion words in that it takes a long time. In addition, English thesaurus-based extraction of Korean opinion words leaves a challenge to reconsider the deterioration of precision attributed to the one to one mismatching between Korean and English words. Studies based on Korean phrase analyzers may potentially fail due to the fact that they select opinion words with a low level of frequency. Therefore, this study will suggest the k-Structure (k=5 or 8) method, which may possibly improve the precision while mutually complementing existing studies in Korea, in automatically extracting opinion words from a simple sentence in a given Korean product review. A simple sentence is defined to be composed of at least 3 words, i.e., a sentence including an opinion word in ${\pm}2$ distance from the attribute name (e.g., the 'battery' of a camera) of a evaluated product (e.g., a 'camera'). In the performance experiment, the precision of those opinion words for 8 previously given attribute names were automatically extracted and estimated for 1,868 product reviews collected from major domestic shopping malls, by using k-Structure. The results showed that k=5 led to a recall of 79.0% and a precision of 87.0%; while k=8 led to a recall of 92.35% and a precision of 89.3%. Also, a test was conducted using PMI-IR (Pointwise Mutual Information - Information Retrieval) out of those methods suggested in English studies, which resulted in a recall of 55% and a precision of 57%.

Product Evaluation Criteria Extraction through Online Review Analysis: Using LDA and k-Nearest Neighbor Approach (온라인 리뷰 분석을 통한 상품 평가 기준 추출: LDA 및 k-최근접 이웃 접근법을 활용하여)

  • Lee, Ji Hyeon;Jung, Sang Hyung;Kim, Jun Ho;Min, Eun Joo;Yeo, Un Yeong;Kim, Jong Woo
    • Journal of Intelligence and Information Systems
    • /
    • v.26 no.1
    • /
    • pp.97-117
    • /
    • 2020
  • Product evaluation criteria is an indicator describing attributes or values of products, which enable users or manufacturers measure and understand the products. When companies analyze their products or compare them with competitors, appropriate criteria must be selected for objective evaluation. The criteria should show the features of products that consumers considered when they purchased, used and evaluated the products. However, current evaluation criteria do not reflect different consumers' opinion from product to product. Previous studies tried to used online reviews from e-commerce sites that reflect consumer opinions to extract the features and topics of products and use them as evaluation criteria. However, there is still a limit that they produce irrelevant criteria to products due to extracted or improper words are not refined. To overcome this limitation, this research suggests LDA-k-NN model which extracts possible criteria words from online reviews by using LDA and refines them with k-nearest neighbor. Proposed approach starts with preparation phase, which is constructed with 6 steps. At first, it collects review data from e-commerce websites. Most e-commerce websites classify their selling items by high-level, middle-level, and low-level categories. Review data for preparation phase are gathered from each middle-level category and collapsed later, which is to present single high-level category. Next, nouns, adjectives, adverbs, and verbs are extracted from reviews by getting part of speech information using morpheme analysis module. After preprocessing, words per each topic from review are shown with LDA and only nouns in topic words are chosen as potential words for criteria. Then, words are tagged based on possibility of criteria for each middle-level category. Next, every tagged word is vectorized by pre-trained word embedding model. Finally, k-nearest neighbor case-based approach is used to classify each word with tags. After setting up preparation phase, criteria extraction phase is conducted with low-level categories. This phase starts with crawling reviews in the corresponding low-level category. Same preprocessing as preparation phase is conducted using morpheme analysis module and LDA. Possible criteria words are extracted by getting nouns from the data and vectorized by pre-trained word embedding model. Finally, evaluation criteria are extracted by refining possible criteria words using k-nearest neighbor approach and reference proportion of each word in the words set. To evaluate the performance of the proposed model, an experiment was conducted with review on '11st', one of the biggest e-commerce companies in Korea. Review data were from 'Electronics/Digital' section, one of high-level categories in 11st. For performance evaluation of suggested model, three other models were used for comparing with the suggested model; actual criteria of 11st, a model that extracts nouns by morpheme analysis module and refines them according to word frequency, and a model that extracts nouns from LDA topics and refines them by word frequency. The performance evaluation was set to predict evaluation criteria of 10 low-level categories with the suggested model and 3 models above. Criteria words extracted from each model were combined into a single words set and it was used for survey questionnaires. In the survey, respondents chose every item they consider as appropriate criteria for each category. Each model got its score when chosen words were extracted from that model. The suggested model had higher scores than other models in 8 out of 10 low-level categories. By conducting paired t-tests on scores of each model, we confirmed that the suggested model shows better performance in 26 tests out of 30. In addition, the suggested model was the best model in terms of accuracy. This research proposes evaluation criteria extracting method that combines topic extraction using LDA and refinement with k-nearest neighbor approach. This method overcomes the limits of previous dictionary-based models and frequency-based refinement models. This study can contribute to improve review analysis for deriving business insights in e-commerce market.

Development of a Korean Speech Recognition Platform (ECHOS) (한국어 음성인식 플랫폼 (ECHOS) 개발)

  • Kwon Oh-Wook;Kwon Sukbong;Jang Gyucheol;Yun Sungrack;Kim Yong-Rae;Jang Kwang-Dong;Kim Hoi-Rin;Yoo Changdong;Kim Bong-Wan;Lee Yong-Ju
    • The Journal of the Acoustical Society of Korea
    • /
    • v.24 no.8
    • /
    • pp.498-504
    • /
    • 2005
  • We introduce a Korean speech recognition platform (ECHOS) developed for education and research Purposes. ECHOS lowers the entry barrier to speech recognition research and can be used as a reference engine by providing elementary speech recognition modules. It has an easy simple object-oriented architecture, implemented in the C++ language with the standard template library. The input of the ECHOS is digital speech data sampled at 8 or 16 kHz. Its output is the 1-best recognition result. N-best recognition results, and a word graph. The recognition engine is composed of MFCC/PLP feature extraction, HMM-based acoustic modeling, n-gram language modeling, finite state network (FSN)- and lexical tree-based search algorithms. It can handle various tasks from isolated word recognition to large vocabulary continuous speech recognition. We compare the performance of ECHOS and hidden Markov model toolkit (HTK) for validation. In an FSN-based task. ECHOS shows similar word accuracy while the recognition time is doubled because of object-oriented implementation. For a 8000-word continuous speech recognition task, using the lexical tree search algorithm different from the algorithm used in HTK, it increases the word error rate by $40\%$ relatively but reduces the recognition time to half.