• Journal of the Korean Data and Information Science Society
• /
• v.27 no.3
• /
• pp.609-619
• /
• 2016

RHadoop is a collection of R packages that allow users to manage and analyze data with Hadoop. In this paper, we implement K-Means algorithm based on MapReduce framework with RHadoop to make the clustering method applicable to large scale data. The main idea introduces a combiner as a function of our map output to decrease the amount of data needed to be processed by reducers. We showed that our K-Means algorithm using RHadoop with combiner was faster than regular algorithm without combiner as the size of data set increases. We also implemented Elbow method with MapReduce for finding the optimum number of clusters for K-Means clustering on large dataset. Comparison with our MapReduce implementation of Elbow method and classical kmeans() in R with small data showed similar results.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.40 no.4
• /
• pp.203-210
• /
• 2017

Data clustering is one of the most difficult and challenging problems and can be formally considered as a particular kind of NP-hard grouping problems. The K-means algorithm is one of the most popular and widely used clustering method because it is easy to implement and very efficient. However, it has high possibility to trap in local optimum and high variation of solutions with different initials for the large data set. Therefore, we need study efficient computational intelligence method to find the global optimal solution in data clustering problem within limited computational time. The objective of this paper is to propose a combined artificial bee colony (CABC) with K-means for initialization and finalization to find optimal solution that is effective on data clustering optimization problem. The artificial bee colony (ABC) is an algorithm motivated by the intelligent behavior exhibited by honeybees when searching for food. The performance of ABC is better than or similar to other population-based algorithms with the added advantage of employing fewer control parameters. Our proposed CABC method is able to provide near optimal solution within reasonable time to balance the converged and diversified searches. In this paper, the experiment and analysis of clustering problems demonstrate that CABC is a competitive approach comparing to previous partitioning approaches in satisfactory results with respect to solution quality. We validate the performance of CABC using Iris, Wine, Glass, Vowel, and Cloud UCI machine learning repository datasets comparing to previous studies by experiment and analysis. Our proposed KABCK (K-means+ABC+K-means) is better than ABCK (ABC+K-means), KABC (K-means+ABC), ABC, and K-means in our simulations.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.17 no.2
• /
• pp.167-172
• /
• 2007

Long gene sequences and their products have been studied by many methods. The use of DNA(Deoxyribonucleic acid) microarray technology has resulted in an enormous amount of data, which has been difficult to analyze using typical research methods. This paper proposes that mass data be analyzed using division clustering with the K-means clustering algorithm. To demonstrate the superiority of the proposed method, it was used to analyze the microarray data from rice DNA. The results were compared to those of the existing K-meansmethod establishing that the proposed method is more useful in spite of the effective reduction of performance time.

• Journal of the Institute of Convergence Signal Processing
• /
• v.5 no.3
• /
• pp.205-209
• /
• 2004

The K-means algorithm is the most widely used method among the codebook generation algorithms in vector quantization. In this paper, we propose a codebook generation algorithm using a new updating condition to enhance the codebook performance. The conventional K-means algorithm uses a fixed weight of the distance for all training iterations, but the proposed method uses different weights according to the updating condition from the new codevectors for training iterations. Then, different weights can be applied to generate codevectors at each iteration according to this condition, and it can have a similar effect to variable weights. Experimental results show that the proposed algorithm has the better codebook performance than that of K-means algorithm.

• Journal of Electrical Engineering and Technology
• /
• v.11 no.6
• /
• pp.1872-1879
• /
• 2016

The techniques to recycle and reuse plastics attract public attention. These public attraction and needs result in improving the recycling technique. However, the identification technique for black plastic wastes still have big problem that the spectrum extracted from near infrared radiation spectroscopy is not clear and is contaminated by noise. To overcome this problem, we apply Raman spectroscopy to extract a clear spectrum of plastic material. In addition, to improve the classification ability of fuzzy Radial Basis Function Neural Networks, we apply supervised learning based clustering method instead of unsupervised clustering method. The conditional fuzzy C-Means clustering method, which is a kind of supervised learning based clustering algorithms, is used to determine the location of radial basis functions. The conditional fuzzy C-Means clustering analyzes the data distribution over input space under the supervision of auxiliary information. The auxiliary information is defined by using k Nearest Neighbor approach.

• Journal of the Institute of Convergence Signal Processing
• /
• v.16 no.1
• /
• pp.9-14
• /
• 2015

Outlier detection and removal is a crucial step needed for various image processing applications such as image registration. Random Sample Consensus (RANSAC) is known to be the best algorithm so far for the outlier detection and removal. However RANSAC requires a cosiderable computation time. To drastically reduce the computation time while preserving the comparable quality, a outlier detection and removal method based on modified K-means is proposed. The original K-means was conducted first for matching point pairs and then cluster merging and member exclusion step are performed in the modification step. We applied the methods to various images with highly repetitive patterns under several geometric distortions and obtained successful results. We compared the proposed method with RANSAC and showed that the proposed method runs 3~10 times faster than RANSAC.

• Proceedings of the Korean Geotechical Society Conference
• /
• 2010.09a
• /
• pp.60-68
• /
• 2010

In this study, the seepage flow monitoring method by hydaulic head loss rate graph was developed for the purpose of monitoring the seepage flow from the see side or from the lake on sea dike in which seepage force was varied periodically. The hydraulic head loss rate was defined in this method. The value of the rate is in the range from 0 to 1. the value of 0 means perfectly free flow of seepage. the value of 1 means perfect waterproofing. The value of coefficient of determination in the hydraulic head loss rate graph closer to 1 means that the seepage flow way is stable. The value of coefficient of determination in the hydraulic head loss rate graph closer to 0 means that the hole may exist or the piping may be in the progress. The pore water pressure data measured in Saemangeum sea dike was analyzed with the developed method The result showed that the variation of seepage flow state was detected sensitively by this method and the interception effect of sea dike could be estimated quantitatively.

• Journal of the Korean Data and Information Science Society
• /
• v.21 no.1
• /
• pp.121-128
• /
• 2010

K-means clustering is an iterative algorithm in which items are moved among sets of clusters until the desired set is reached. K-means clustering has been widely used in many applications, such as market research, pattern analysis or recognition, image processing, etc. It can identify dense and sparse regions among data attributes or object attributes. But k-means algorithm requires many hours to get k clusters that we want, because it is more primitive, explorative. In this paper we propose a new method of k-means clustering using a center of gravity for grid-based sample. It is more fast than any traditional clustering method and maintains its accuracy.

• Journal of Korea Multimedia Society
• /
• v.17 no.10
• /
• pp.1160-1170
• /
• 2014

Clustering methods are very useful in many fields such as data mining, classification, and object recognition. Both the supervised and unsupervised grouping approaches can classify a series of sample data with a predefined or automatically assigned cluster number. However, there is no constraint on the number of elements for each cluster. Numbers of cluster members for each cluster obtained from clustering schemes are usually random. Thus, some clusters possess a large number of elements whereas others only have a few members. In some areas such as logistics management, a fixed number of members are preferred for each cluster or logistic center. Consequently, it is necessary to design a clustering method that can automatically adjust the number of group elements. In this paper, a k-means based clustering method with a fixed number of cluster members is proposed. In the proposed method, first, the data samples are clustered using the k-means algorithm. Then, the number of group elements is adjusted by employing a greedy strategy. Experimental results demonstrate that the proposed clustering scheme can classify data samples efficiently for a fixed number of cluster members.

• Communications for Statistical Applications and Methods
• /
• v.12 no.2
• /
• pp.531-538
• /
• 2005

In k-means clustering, we standardize variables before clustering and iterate two steps: units allocation by Euclidean sense and centroids updating. In applications to DB marketing where clusters are to be used as customer segments with similar consumption behaviors, we frequently acquire additional variables on the customers or the units through marketing campaigns a posteriori. Hence we need to modify the clusters originally formed after each campaign. The aim of this study is to propose a revision method of k-means clusters, incorporating added information by weighting clustering variables. We illustrate the proposed method in an empirical case.