• 제목/요약/키워드: Pre-Clustering

검색결과 124건 처리시간 0.023초

대용량 데이터 처리를 위한 하이브리드형 클러스터링 기법 (A Hybrid Clustering Technique for Processing Large Data)

  • 김만선;이상용
    • 정보처리학회논문지B
    • /
    • 제10B권1호
    • /
    • pp.33-40
    • /
    • 2003
  • 데이터 마이닝은 지식발견 과정에서 중요한 역할을 수행하며, 여러 데이터 마이닝의 알고리즘들은 특정의 목적을 위하여 선택될 수 있다. 대부분의 전통적인 계층적 클러스터링 방법은 적은 양의 데이터 집합을 처리하는데 적합하여 제한된 리소스와 부족한 효율성으로 인하여 대용량의 데이터 집합을 다루기가 곤란하다. 본 연구에서는 대용량의 데이터에 적용되어 알려지지 않은 패턴을 발견할 수 있는 하이브리드형 신경망 클러스터링 기법의 PPC(Pre-Post Clustrering) 기법을 제안한다. PPC 기법은 인공지능적 방법인 자기조직화지도(SOM)와 통계적 방법인 계층적 클러스터링을 결합하여 두 과정에서는 군집의 내부적 특징을 나타내는 응집거리와 군집간의 외부적 거리를 나타내는 인접거리에 따라 유사도를 측정한다. 최종적으로 PPC 기법은 측정된 유사도를 이용하여 대용량 데이터 집합을 군집화한다. PPC 기법은 UCI Repository 데이터를 이용하여 실험해 본 결과, 다른 클러스터링 기법들 보다 우수한 응집도를 보였다.

Pre-Adjustment of Incomplete Group Variable via K-Means Clustering

  • Hwang, S.Y.;Hahn, H.E.
    • Journal of the Korean Data and Information Science Society
    • /
    • 제15권3호
    • /
    • pp.555-563
    • /
    • 2004
  • In classification and discrimination, we often face with incomplete group variable arising typically from many missing values and/or incredible cases. This paper suggests the use of K-means clustering for pre-adjusting incompleteness and in turn classification based on generalized statistical distance is performed. For illustrating the proposed procedure, simulation study is conducted comparatively with CART in data mining and traditional techniques which are ignoring incompleteness of group variable. Simulation study manifests that our methodology out-performs.

  • PDF

A Simple Tandem Method for Clustering of Multimodal Dataset

  • Cho C.;Lee J.W.;Lee J.W.
    • 한국경영과학회:학술대회논문집
    • /
    • 한국경영과학회/대한산업공학회 2003년도 춘계공동학술대회
    • /
    • pp.729-733
    • /
    • 2003
  • The presence of local features within clusters incurred by multi-modal nature of data prohibits many conventional clustering techniques from working properly. Especially, the clustering of datasets with non-Gaussian distributions within a cluster can be problematic when the technique with implicit assumption of Gaussian distribution is used. Current study proposes a simple tandem clustering method composed of k-means type algorithm and hierarchical method to solve such problems. The multi-modal dataset is first divided into many small pre-clusters by k-means or fuzzy k-means algorithm. The pre-clusters found from the first step are to be clustered again using agglomerative hierarchical clustering method with Kullback- Leibler divergence as the measure of dissimilarity. This method is not only effective at extracting the multi-modal clusters but also fast and easy in terms of computation complexity and relatively robust at the presence of outliers. The performance of the proposed method was evaluated on three generated datasets and six sets of publicly known real world data.

  • PDF

군집분석을 이용한 침수관련 유역특성 분류 (Classification of basin characteristics related to inundation using clustering)

  • 이한승;조재웅;강호선;황정근;문혜진
    • 한국수자원학회:학술대회논문집
    • /
    • 한국수자원학회 2020년도 학술발표회
    • /
    • pp.96-96
    • /
    • 2020
  • In order to establish the risk criteria of inundation due to typhoons or heavy rainfall, research is underway to predict the limit rainfall using basin characteristics, limit rainfall and artificial intelligence algorithms. In order to improve the model performance in estimating the limit rainfall, the learning data are used after the pre-processing. When 50.0% of the entire data was removed as an outlier in the pre-processing process, it was confirmed that the accuracy is over 90%. However, the use rate of learning data is very low, so there is a limitation that various characteristics cannot be considered. Accordingly, in order to predict the limit rainfall reflecting various watershed characteristics by increasing the use rate of learning data, the watersheds with similar characteristics were clustered. The algorithms used for clustering are K-Means, Agglomerative, DBSCAN and Spectral Clustering. The k-Means, DBSCAN and Agglomerative clustering algorithms are clustered at the impervious area ratio, and the Spectral clustering algorithm is clustered in various forms depending on the parameters. If the results of the clustering algorithm are applied to the limit rainfall prediction algorithm, various watershed characteristics will be considered, and at the same time, the performance of predicting the limit rainfall will be improved.

  • PDF

Semantic Correspondence of Database Schema from Heterogeneous Databases using Self-Organizing Map

  • Dumlao, Menchita F.;Oh, Byung-Joo
    • 전기전자학회논문지
    • /
    • 제12권4호
    • /
    • pp.217-224
    • /
    • 2008
  • This paper provides a framework for semantic correspondence of heterogeneous databases using self- organizing map. It solves the problem of overlapping between different databases due to their different schemas. Clustering technique using self-organizing maps (SOM) is tested and evaluated to assess its performance when using different kinds of data. Preprocessing of database is performed prior to clustering using edit distance algorithm, principal component analysis (PCA), and normalization function to identify the features necessary for clustering.

  • PDF

이동물체 탐지를 위한 레이다 데이터의 거리-도플러 클러스터링 기법 (Range-Doppler Clustering of Radar Data for Detecting Moving Objects)

  • 김성준;양동원;정영헌;김수진;윤주홍
    • 한국군사과학기술학회지
    • /
    • 제17권6호
    • /
    • pp.810-820
    • /
    • 2014
  • Recently many studies of Radar systems mounted on ground vehicles for autonomous driving, SLAM (Simultaneous localization and mapping) and collision avoidance are reported. In near field, several hits per an object are generated after signal processing of Radar data. Hence, clustering is an essential technique to estimate their shapes and positions precisely. This paper proposes a method of grouping hits in range-doppler domains into clusters which represent each object, according to the pre-defined rules. The rules are based on the perceptual cues to separate hits by object. The morphological connectedness between hits and the characteristics of SNR distribution of hits are adopted as the perceptual cues for clustering. In various simulations for the performance assessment, the proposed method yielded more effective performance than other techniques.

신경망을 사용한 사상체질 진단검사 개발 연구 (Development of Sasang Type Diagnostic Test with Neural Network)

  • 채한;황상문;엄일규;김병철;김영인;김병주;권영규
    • 동의생리병리학회지
    • /
    • 제23권4호
    • /
    • pp.765-771
    • /
    • 2009
  • The medical informatics for clustering Sasang types with collected clinical data is important for the personalized medicine, but it has not been thoroughly studied yet. The purpose of this study was to examine the usefulness of neural network data mining algorithm for traditional Korean medicine. We used Kohonen neural network, the Self-Organizing Map (SOM), for the analysis of biomedical information following data pre-processing and calculated the validity index as percentage correctly predicted and type-specific sensitivity. We can extract 12 data fields from 30 after data pre-processing with correlation analysis and latent functional relationship analysis. The profile of Myers-Briggs Type Inidcator and Bio-Impedance Analysis data which are clustered with SOM was similar to that of original measurements. The percentage correctly predicted was 56%, and sensitivity for So-Yang, Tae-Eum and So-Eum type were 56%, 48%, and 61%, respectively. This study showed that the neural network algorithm for clustering Sasang types based on clinical data is useful for the sasang type diagnostic test itself. We discussed the importance of data pre-processing and clustering algorithm for the validity of medical devices in traditional Korean medicine.

Pixel 군집화 Data를 이용한 실시간 반사광 검출 알고리즘 (Real-time Reflection Light Detection Algorithm using Pixel Clustering Data)

  • 황도경;안종우;강호선;이장명
    • 로봇학회논문지
    • /
    • 제14권4호
    • /
    • pp.301-310
    • /
    • 2019
  • A new algorithm has been propose to detect the reflected light region as disturbances in a real-time vision system. There have been several attempts to detect existing reflected light region. The conventional mathematical approach requires a lot of complex processes so that it is not suitable for a real-time vision system. On the other hand, when a simple detection process has been applied, the reflected light region can not be detected accurately. Therefore, in order to detect reflected light region for a real-time vision system, the detection process requires a new algorithm that is as simple and accurate as possible. In order to extract the reflected light, the proposed algorithm has been adopted several filter equations and clustering processes in the HSI (Hue Saturation Intensity) color space. Also the proposed algorithm used the pre-defined reflected light data generated through the clustering processes to make the algorithm simple. To demonstrate the effectiveness of the proposed algorithm, several images with the reflected region have been used and the reflected regions are detected successfully.

Online Clustering Algorithms for Semantic-Rich Network Trajectories

  • Roh, Gook-Pil;Hwang, Seung-Won
    • Journal of Computing Science and Engineering
    • /
    • 제5권4호
    • /
    • pp.346-353
    • /
    • 2011
  • With the advent of ubiquitous computing, a massive amount of trajectory data has been published and shared in many websites. This type of computing also provides motivation for online mining of trajectory data, to fit user-specific preferences or context (e.g., time of the day). While many trajectory clustering algorithms have been proposed, they have typically focused on offline mining and do not consider the restrictions of the underlying road network and selection conditions representing user contexts. In clear contrast, we study an efficient clustering algorithm for Boolean + Clustering queries using a pre-materialized and summarized data structure. Our experimental results demonstrate the efficiency and effectiveness of our proposed method using real-life trajectory data.

Inter-Cloud 환경을 위한 IAM 클러스터링 아키텍처 (IAM Clustering Architecture for Inter-Cloud Environment)

  • 김진욱;박정수;박민호;정수환
    • 한국통신학회논문지
    • /
    • 제40권5호
    • /
    • pp.860-862
    • /
    • 2015
  • 본 논문에서는 Inter-Cloud 환경에서 효율적인 사용자 인증 및 권한 인가를 위한 새로운 형태의 IAM 클러스터링 아키텍처를 제안한다. 제안하는 클러스터링 아키텍처는 사전 Access Agreement를 통하여 사용자가 자신이 등록되지 않은 어떤 서비스도 간단하게 이용할 수 있도록 인증 및 접근 권한을 제공한다. 본 논문에서는 IAM 클러스터링 아키텍처의 구성요소 및 인증 프로토콜을 설명한다.