• Title/Summary/Keyword: 가중치 마이닝

Search Result 115, Processing Time 0.029 seconds

A data mining approach for efficient matching of engineering document schemata (엔지니어링 문서 스키마의 효율적 매칭을 위한 데이터마이닝 기법의 활용방안)

  • Park, Sang-Il;An, Hyun-Jung;Kim, Hyo-Jin;Lee, Sang-Ho
    • Proceedings of the Computational Structural Engineering Institute Conference
    • /
    • 2010.04a
    • /
    • pp.226-229
    • /
    • 2010
  • 본 연구에서는 데이터 저장의 질적 향상을 도모하는 XML 스키마 매칭의 효율적 활용방안을 제시하였다. 이를 위하여 매칭의 가중치의 변화에 따라 달라지는 정확도 데이터를 수집하고, 수집한 데이터를 활용하여 데이터 마이닝 기법 중 하나인 의사결정나무 모델을 수립하였다. 수립모델을 응용하여 구현한 가중치 자동선정 모듈은 설명변수인 교량의 형식, 문서가 포함하고 있는 요소의 수, 문서를 작성한 회사 등의 값에 따라 의사결정나무 모델의 목표변수인 정확도뿐만 아니라, 가장 높은 정확도를 보일 수 있는 가중치까지 간접적으로 제안가능하다. 본 연구로 구현한 모듈을 통해 제안된 XML 스키마 매칭 가중치를 활용하면 그렇지 않은 경우에 비하여 약 10% 정확도 상승효과가 있음을 알 수 있었다.

  • PDF

Efficient Dynamic Weighted Frequent Pattern Mining by using a Prefix-Tree (Prefix-트리를 이용한 동적 가중치 빈발 패턴 탐색 기법)

  • Jeong, Byeong-Soo;Farhan, Ahmed
    • The KIPS Transactions:PartD
    • /
    • v.17D no.4
    • /
    • pp.253-258
    • /
    • 2010
  • Traditional frequent pattern mining considers equal profit/weight value of every item. Weighted Frequent Pattern (WFP) mining becomes an important research issue in data mining and knowledge discovery by considering different weights for different items. Existing algorithms in this area are based on fixed weight. But in our real world scenarios the price/weight/importance of a pattern may vary frequently due to some unavoidable situations. Tracking these dynamic changes is very necessary in different application area such as retail market basket data analysis and web click stream management. In this paper, we propose a novel concept of dynamic weight and an algorithm DWFPM (dynamic weighted frequent pattern mining). Our algorithm can handle the situation where price/weight of a pattern may vary dynamically. It scans the database exactly once and also eligible for real time data processing. To our knowledge, this is the first research work to mine weighted frequent patterns using dynamic weights. Extensive performance analyses show that our algorithm is very efficient and scalable for WFP mining using dynamic weights.

Frequent Pattern Mining By using a Completeness for BigData (빅데이터에 대한 Completeness를 이용한 빈발 패턴 마이닝)

  • Park, In-Kyu
    • Journal of Korea Game Society
    • /
    • v.18 no.2
    • /
    • pp.121-130
    • /
    • 2018
  • Most of those studies use frequency, the number of times a pattern appears in a transaction database, as the key measure for pattern interestingness. It prerequisites that any interesting pattern should occupy a maximum portion of the transactions it appears. But in our real world scenarios the completeness of any pattern is more likely to become various in transactions. Hence, we should also consider the problem of finding the qualified patterns with the significant values of the weighted support by completeness in order to reduce the loss of information within any pattern in transaction. In these pattern recommendation applications, patterns with higher completeness may lead to higher recall while patterns with higher completeness may lead to higher recall while patterns with higher frequency lead to higher precision. In this paper, we propose a measure of weighted support and completeness and an algorithm WSCFPM(weigted support and completeness frequent pattern mining). Our algorithm handles the invalidation of the monotone or anti-monotone property which does not hold on completeness. Extensive performance analysis show that our algorithm is very efficient and scalable for word pattern mining.

Adaptive Frequent Pattern Algorithm using CAWFP-Tree based on RHadoop Platform (RHadoop 플랫폼기반 CAWFP-Tree를 이용한 적응 빈발 패턴 알고리즘)

  • Park, In-Kyu
    • Journal of Digital Convergence
    • /
    • v.15 no.6
    • /
    • pp.229-236
    • /
    • 2017
  • An efficient frequent pattern algorithm is essential for mining association rules as well as many other mining tasks for convergence with its application spread over a very broad spectrum. Models for mining pattern have been proposed using a FP-tree for storing compressed information about frequent patterns. In this paper, we propose a centroid frequent pattern growth algorithm which we called "CAWFP-Growth" that enhances he FP-Growth algorithm by making the center of weights and frequencies for the itemsets. Because the conventional constraint of maximum weighted support is not necessary to maintain the downward closure property, it is more likely to reduce the search time and the information loss of the frequent patterns. The experimental results show that the proposed algorithm achieves better performance than other algorithms without scarifying the accuracy and increasing the processing time via the centroid of the items. The MapReduce framework model is provided to handle large amounts of data via a pseudo-distributed computing environment. In addition, the modeling of the proposed algorithm is required in the fully distributed mode.

A Sequential Pattern Mining based on Dynamic Weight in Data Stream (스트림 데이터에서 동적 가중치를 이용한 순차 패턴 탐사 기법)

  • Choi, Pilsun;Kim, Hwan;Kim, Daein;Hwang, Buhyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.137-144
    • /
    • 2013
  • A sequential pattern mining is finding out frequent patterns from the data set in time order. In this field, a dynamic weighted sequential pattern mining is applied to a computing environment that changes depending on the time and it can be utilized in a variety of environments applying changes of dynamic weight. In this paper, we propose a new sequence data mining method to explore the stream data by applying the dynamic weight. This method reduces the candidate patterns that must be navigated by using the dynamic weight according to the relative time sequence, and it can find out frequent sequence patterns quickly as the data input and output using a hash structure. Using this method reduces the memory usage and processing time more than applying the existing methods. We show the importance of dynamic weighted mining through the comparison of different weighting sequential pattern mining techniques.

Disease related Gene Identification Using Literature and Google data (텍스트마이닝 기법과 구글데이터를 이용한 질병관련 유전자 식별)

  • Kim, Jeong-U;Kim, Hyeon-Jin;Park, Sang-Hyeon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.11a
    • /
    • pp.1084-1087
    • /
    • 2013
  • 텍스트마이닝은(Text mining) 바이오분야에서 사용되는 도구 중 하나이다. 본 논문에서는 전립선암(Prostate cancer)과 관련된 질병 유전자(Disease gene)를 찾기 위해 텍스트마이닝을 이용하여 유전자 네트워크(Gene-network)를 구축하였다. 추가적으로 구글(Google) 검색을 통해 네트워크 내의 유전자 노드(Node)들 사이의 간선(Edge)에 새로운 가중치(Weight)를 추가하고 네트워크를 재구성하였다. 구축된 네트워크에서 노드와 노드 사이의 가중치를 기반으로 전립선암과 관련된 질병 유전자를 추출하였다. 본 논문의 방법은 성공적으로 네트워크를 구축하고 질병 유전자를 찾았으며, 구글 데이터를 사용하지 않고 네트워크를 구축하는 경우보다 더 높은 정확성을 입증했다.

A New Learning Algorithm for Rare Class Classification (희귀 목적값 분류를 위한 학습 알고리즘)

  • Lee, Kwang-Ho;Lee, Chang-Hwan
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2006.11a
    • /
    • pp.39-42
    • /
    • 2006
  • 본 논문에서는 데이터 마이닝에서 발생되는 희귀 데이터를 분석하기 위한 희귀 목적값 분석의 새로운 알고리즘을 제시한다. 이를 위하여 속성들이 가지는 속성의 가중치 값과 속성값이 목적 속성에 미치는 가중치값을 정보이론에 입각하여 가중치 계산을 하고, 계산된 가중치값을 사용하여 스코어링 함으로써 희귀 목적값에 속한 데이터 예측/분류에 사용하는 방법을 제시하였다. 실험을 통해 본 알고리즘의 성능을 입증함은 물론 제안된 알고리즘이 희귀 데이터의 분류/학습에 좀 더 효과적이다는 것을 보였다.

  • PDF

Recent Technique Analysis, Infant Commodity Pattern Analysis Scenario and Performance Analysis of Incremental Weighted Maximal Representative Pattern Mining (점진적 가중화 맥시멀 대표 패턴 마이닝의 최신 기법 분석, 유아들의 물품 패턴 분석 시나리오 및 성능 분석)

  • Yun, Unil;Yun, Eunmi
    • Journal of Internet Computing and Services
    • /
    • v.21 no.2
    • /
    • pp.39-48
    • /
    • 2020
  • Data mining techniques have been suggested to find efficiently meaningful and useful information. Especially, in the big data environments, as data becomes accumulated in several applications, related pattern mining methods have been proposed. Recently, instead of analyzing not only static data stored already in files or databases, mining dynamic data incrementally generated in a real time is considered as more interesting research areas because these dynamic data can be only one time read. With this reason, researches of how these dynamic data are mined efficiently have been studied. Moreover, approaches of mining representative patterns such as maximal pattern mining have been proposed since a huge number of result patterns as mining results are generated. As another issue, to discover more meaningful patterns in real world, weights of items in weighted pattern mining have been used, In real situation, profits, costs, and so on of items can be utilized as weights. In this paper, we analyzed weighted maximal pattern mining approaches for data generated incrementally. Maximal representative pattern mining techniques, and incremental pattern mining methods. And then, the application scenarios for analyzing the required commodity patterns in infants are presented by applying weighting representative pattern mining. Furthermore, the performance of state-of-the-art algorithms have been evaluated. As a result, we show that incremental weighted maximal pattern mining technique has better performance than incremental weighted pattern mining and weighted maximal pattern mining.

High Utility Pattern Mining using a Prefix-Tree (Prefix-Tree를 이용한 높은 유틸리티 패턴 마이닝 기법)

  • Jeong, Byeong-Soo;Ahmed, Chowdhury Farhan;Lee, In-Gi;Yong, Hwan-Seong
    • Journal of KIISE:Databases
    • /
    • v.36 no.5
    • /
    • pp.341-351
    • /
    • 2009
  • Recently high utility pattern (HUP) mining is one of the most important research issuer in data mining since it can consider the different weight Haloes of items. However, existing mining algorithms suffer from the performance degradation because it cannot easily apply Apriori-principle for pattern mining. In this paper, we introduce new high utility pattern mining approach by using a prefix-tree as in FP-Growth algorithm. Our approach stores the weight value of each item into a node and utilizes them for pruning unnecessary patterns. We compare the performance characteristics of three different prefix-tree structures. By thorough experimentation, we also prove that our approach can give performance improvement to a degree.

Assessing Classification Accuracy using Cohen's kappa in Data Mining (데이터 마이닝에서 Cohen의 kappa를 이용한 분류정확도 측정)

  • Um, Yonghwan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.18 no.1
    • /
    • pp.177-183
    • /
    • 2013
  • In this paper, Cohen's kappa and weighted kappa are applied to measuring classification accuracy when performing classification in data minig. Cohen's kappa compensates for classifications that may be due to chance and is used for the data with nominal or ordinal scales. Especially, for the ordinal data, weighted kappa which measures the classification accuracy by quantifying the classification errors as weights is used. We used two weights (linear weight, quadratic weight) for calculations of weighted kappa. Also for the calculation and comparison of kappa and weighted kappa we used a real data set, fat-liver data.