• 제목/요약/키워드: Text Mining for Korean

검색결과 642건 처리시간 0.026초

텍스트마이닝을 활용한 건설분야 트랜드 분석 (Analysis of trend in construction using textmining method)

  • 정철우;김재준
    • 한국디지털건축인테리어학회논문집
    • /
    • 제12권2호
    • /
    • pp.53-60
    • /
    • 2012
  • In this paper, we present new methods for identifying keywords for foresight topics that utilize the internet and textmining techniques to draw objective and quantified information that support experts' qualitative opinions and evaluations in foresight. Furthermore, by applying this fabricated procedure, we have derived keywords to analyze priorities in architectural engineering. Not much difference between qualitative methods of experts and quantitative methods such as text mining has been observed from comparison between technologies derived via qualitative method from "The Science Technology Vision" (control group). Therefore, as a quantitative tool useful for drawing keywords for foresight, textmining can supplement quantitative analysis by experts. In addition, depending on the level and type of raw data, text mining can bring better results in deriving foresight keywords. For this reason, research activities accommodating Internet search results and the development of textmining methods for analyzing current trends are in demand.

공격 메일 식별을 위한 비정형 데이터를 사용한 유전자 알고리즘 기반의 특징선택 알고리즘 (Feature-selection algorithm based on genetic algorithms using unstructured data for attack mail identification)

  • 홍성삼;김동욱;한명묵
    • 인터넷정보학회논문지
    • /
    • 제20권1호
    • /
    • pp.1-10
    • /
    • 2019
  • 빅 데이터에서 텍스트 마이닝은 많은 수의 데이터로부터 많은 특징 추출하기 때문에, 클러스터링 및 분류 과정의 계산 복잡도가 높고 분석결과의 신뢰성이 낮아질 수 있다. 특히 텍스트마이닝 과정을 통해 얻는 Term document matrix는 term과 문서간의 특징들을 표현하고 있지만, 희소행렬 형태를 보이게 된다. 본 논문에서는 탐지모델을 위해 텍스트마이닝에서 개선된 GA(Genetic Algorithm)을 이용한 특징 추출 방법을 설계하였다. TF-IDF는 특징 추출에서 문서와 용어간의 관계를 반영하는데 사용된다. 반복과정을 통해 사전에 미리 결정된 만큼의 특징을 선택한다. 또한 탐지모델의 성능 향상을 위해 sparsity score(희소성 점수)를 사용하였다. 스팸메일 세트의 희소성이 높으면 탐지모델의 성능이 낮아져 최적화된 탐지 모델을 찾기가 어렵다. 우리는 fitness function에서 s(F)를 사용하여 희소성이 낮고 TF-IDF 점수가 높은 탐지모델을 찾았다. 또한 제안된 알고리즘을 텍스트 분류 실험에 적용하여 성능을 검증하였다. 결과적으로, 제안한 알고리즘은 공격 메일 분류에서 좋은 성능(속도와 정확도)을 보여주었다.

Exploring an Optimal Feature Selection Method for Effective Opinion Mining Tasks

  • Eo, Kyun Sun;Lee, Kun Chang
    • 한국컴퓨터정보학회논문지
    • /
    • 제24권2호
    • /
    • pp.171-177
    • /
    • 2019
  • This paper aims to find the most effective feature selection method for the sake of opinion mining tasks. Basically, opinion mining tasks belong to sentiment analysis, which is to categorize opinions of the online texts into positive and negative from a text mining point of view. By using the five product groups dataset such as apparel, books, DVDs, electronics, and kitchen, TF-IDF and Bag-of-Words(BOW) fare calculated to form the product review feature sets. Next, we applied the feature selection methods to see which method reveals most robust results. The results show that the stacking classifier based on those features out of applying Information Gain feature selection method yields best result.

『동의보감(東醫寶鑑)』에서 항노화 효능을 가진 복방(複方) 후보군의 선별에 대한 연구 (Anti-aging herbal formulae in Dongeuibogam : Identification of candidates by text mining)

  • 배승빈;윤병철;백진웅
    • 대한한의학원전학회지
    • /
    • 제28권4호
    • /
    • pp.1-9
    • /
    • 2015
  • Objectives : The main aims of this study were to identify candidate anti-aging herbal formulae(CAHF) from Dongeuibogam. Methods : We analyzed the terms describing effect of 3,901 herbal formulae on Dongeuibogam and selected the terms describing the anti-aging effect of herbal formulae(TAEHF). Finally, we generated a list of CAHFs based on TAEHFs. Results & Conclusions : 1. We finally selected 162 TAEHFs on Dongeuibogam. 2. We finally selected 138 CAHFs on Dongeuibogam(15CAHFs for external use, 123CAHFs for internal use). 3. TAEHFs are classified into 9 types. 4. CAHFs are classified into 9 types.

IMPLEMENTATION OF SUBSEQUENCE MAPPING METHOD FOR SEQUENTIAL PATTERN MINING

  • Trang, Nguyen Thu;Lee, Bum-Ju;Lee, Heon-Gyu;Ryu, Keun-Ho
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 2006년도 Proceedings of ISRS 2006 PORSEC Volume II
    • /
    • pp.627-630
    • /
    • 2006
  • Sequential Pattern Mining is the mining approach which addresses the problem of discovering the existent maximal frequent sequences in a given databases. In the daily and scientific life, sequential data are available and used everywhere based on their representative forms as text, weather data, satellite data streams, business transactions, telecommunications records, experimental runs, DNA sequences, histories of medical records, etc. Discovering sequential patterns can assist user or scientist on predicting coming activities, interpreting recurring phenomena or extracting similarities. For the sake of that purpose, the core of sequential pattern mining is finding the frequent sequence which is contained frequently in all data sequences. Beside the discovery of frequent itemsets, sequential pattern mining requires the arrangement of those itemsets in sequences and the discovery of which of those are frequent. So before mining sequences, the main task is checking if one sequence is a subsequence of another sequence in the database. In this paper, we implement the subsequence matching method as the preprocessing step for sequential pattern mining. Matched sequences in our implementation are the normalized sequences as the form of number chain. The result which is given by this method is the review of matching information between input mapped sequences.

  • PDF

Sustainable Industry-Academia-Government Collaborative Education Focusing on Advantages of Industry: Long-term Internship after 5years Practice

  • Morimoto, Emi;Yamanaka, Hideo
    • 공학교육연구
    • /
    • 제15권5호
    • /
    • pp.47-53
    • /
    • 2012
  • Practical problem-solving studies in a company or organization have provided great advantages for our university and students. For example, such studies can lead them to build a stronger relationship with local governments and companies as well as develop their research through collaborative studies. On the other hand, comments from companies or organizations that accepted our students showed that they did not always have advantages. This study seeks ways to establish a sustainable long-term internship program that can offer advantages for companies. Advantages and disadvantages of the internship are written by the company on the evaluated sheet. These feedback comments are analyzed by text-mining approach. It is shown that there are three types of company and organizations depending on their reasons for accepting students. Next, suitable internship programs for each type, including their period and expense distribution are presented.

Guiding Practical Text Classification Framework to Optimal State in Multiple Domains

  • Choi, Sung-Pil;Myaeng, Sung-Hyon;Cho, Hyun-Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제3권3호
    • /
    • pp.285-307
    • /
    • 2009
  • This paper introduces DICE, a Domain-Independent text Classification Engine. DICE is robust, efficient, and domain-independent in terms of software and architecture. Each module of the system is clearly modularized and encapsulated for extensibility. The clear modular architecture allows for simple and continuous verification and facilitates changes in multiple cycles, even after its major development period is complete. Those who want to make use of DICE can easily implement their ideas on this test bed and optimize it for a particular domain by simply adjusting the configuration file. Unlike other publically available tool kits or development environments targeted at general purpose classification models, DICE specializes in text classification with a number of useful functions specific to it. This paper focuses on the ways to locate the optimal states of a practical text classification framework by using various adaptation methods provided by the system such as feature selection, lemmatization, and classification models.

Paying Back to Good Deeds: A Text Mining Approach to Explore Don-jjul as Pro-consumption Behavior

  • Hojin Choo;Sue Hyun Lee
    • Asia Marketing Journal
    • /
    • 제26권2호
    • /
    • pp.104-128
    • /
    • 2024
  • More consumers are choosing pro-consumption for social change, but scholars know little about why and how consumers engage in pro-consumption behaviors. A newly emerged pro-consumption behavior called "Don-jjul," which appeared during the COVID-19 pandemic in South Korea, refers to compensating businesses that have engaged in altruistic actions by boosting their sales. This study used Latent Dirichlet Allocation (LDA) of topic modeling, sentiment analysis, and in-depth interviews to investigate the perceptions, motivations, and emotions regarding Don-jjul. As a result, the study revealed pro-consumers' perceptions of Don-jjul as "collective pro-consumption for contributing to social well-being." Don-jjul has two main motives: "supporting underdogs with difficulties" and "compensating good businesses economically." We also found two ambivalent emotions evoked by Don-jjul: "respect for good business owners" and "concerns regarding the misuse of Don-jjul." The results contribute to pro-consumption research for social well-being, providing business opportunities for retailers and CSR managers with a deep understanding of pro-consumers.

문서 분류를 위한 용어 가중치 기법 비교 (Comparison of term weighting schemes for document classification)

  • 정호영;신상민;최용석
    • 응용통계연구
    • /
    • 제32권2호
    • /
    • pp.265-276
    • /
    • 2019
  • 문서-용어 빈도행렬은 텍스트 마이닝에서 분석하고자 하는 개체 정보를 가지고 있는 일반적인 자료 형태이다. 본 연구에서 문서 분류를 위해 문서-용어 빈도행렬에 적용되는 기존의 용어 가중치인 TF-IDF를 소개한다. 추가하여 최근에 알려진 용어 가중치인 TF-IDF-ICSDF와 TF-IGM의 정의와 장단점을 소개하고 비교한다. 또한 문서 분류 분석의 질을 높이기 위해 핵심어를 추출하는 방법을 제시하고자 한다. 추출된 핵심어를 바탕으로 문서 분류에 있어서 가장 많이 활용된 기계학습 알고리즘 중에서 서포트 벡터 머신을 이용하였다. 본 연구에서 소개한 용어 가중치들의 성능을 비교하기 위하여 정확률, 재현율, F1-점수와 같은 성능 지표들을 이용하였다. 그 결과 TF-IGM 방법이 모두 높은 성능 지표를 보였고, 텍스트를 분류하는데 있어 최적화 된 방법으로 나타났다.

온톨로지와 텍스트 마이닝 기반 지능형 역사인물 검색 서비스 (Ontology and Text Mining-based Advanced Historical People Finding Service)

  • 정도헌;황명권;조민희;정한민;윤소영;김경선;김평
    • 인터넷정보학회논문지
    • /
    • 제13권5호
    • /
    • pp.33-43
    • /
    • 2012
  • 시맨틱 웹 기술은 특정 개체를 중심으로 의미적 연관 관계를 생성하고 연관 관계를 이용해서 다양한 지능형 정보 서비스를 구축하는데 활용되며, 텍스트 마이닝 기술은 비정형 데이터를 대상으로 의미 분석을 통해서 의미적 연관 관계를 생성하는데 활용될 수 있다. 본 연구에서는 역사인물을 중심으로 온톨로지 스키마, 인스턴스를 생성하는 가이드라인, 인스턴스 생성, 동명이인 해소를 위한 텍스트 마이닝, 추론을 활용한 지능화된 역사인물 검색서비스를 제안한다. 역사분야 전문가들이 생성한 역사적 사건, 기관, 인물 중심의 연관 관계와 국사편찬위원회에서 보유한 다양한 문헌들 간의 연계를 통해, 사용자들의 정보접근성을 향상시킴과 동시에 관계 정보에 기반한 새로운 역사인물 검색 서비스를 제안하였다. 새로운 역사인물 검색 서비스는 인물간의 소셜 네트워크를 사용하여 역사문헌에 나타난 동명이인을 해소함으로써 보다 정확한 검색서비스를 제공하는 것은 물론, 역사 인물 시소러스를 포함한 다양한 외부 정보와의 연계를 통해서 역사인물에 대한 고부가 정보를 제공하고 있다.