• Title/Summary/Keyword: parallel clustering

Search Result 105, Processing Time 0.033 seconds

An Advanced Parallel Join Algorithm for Managing Data Skew on Hypercube Systems (하이퍼큐브 시스템에서 데이타 비대칭성을 고려한 향상된 병렬 결합 알고리즘)

  • 원영선;홍만표
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.30 no.3_4
    • /
    • pp.117-129
    • /
    • 2003
  • In this paper, we propose advanced parallel join algorithm to efficiently process join operation on hypercube systems. This algorithm uses a broadcasting method in processing relation R which is compatible with hypercube structure. Hence, we can present optimized parallel join algorithm for that hypercube structure. The proposed algorithm has a complete solution of two essential problems - load balancing problem and data skew problem - in parallelization of join operation. In order to solve these problems, we made good use of the characteristics of clustering effect in the algorithm. As a result of this, performance is improved on the whole system than existing algorithms. Moreover. new algorithm has an advantage that can implement non-equijoin operation easily which is difficult to be implemented in hash based algorithm. Finally, according to the cost model analysis. this algorithm showed better performance than existing parallel join algorithms.

EPR : Enhanced Parallel R-tree Indexing Method for Geographic Information System (EPR : 지리 정보 시스템을 위한 향상된 병렬 R-tree 색인 기법)

  • Lee, Chun-Geun;Kim, Jeong-Won;Kim, Yeong-Ju;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.9
    • /
    • pp.2294-2304
    • /
    • 1999
  • Our research purpose in this paper is to improve the performance of query processing in GIS(Geographic Information System) by enhancing the I/O performance exploiting parallel I/O and efficient disk access. By packing adjacent spatial data, which are very likely to be referenced concurrently, into one block or continuous disk blocks, the number of disk accesses and the disk access overhead for query processing can be decreased, and this eventually leads to the I/O time decrease. So, in this paper, we proposes EPR(Enhanced Parallel R-tree) indexing method which integrates the parallel I/O method of the previous Parallel R-tree method and a packing-based clustering method. The major characteristics of EPR method are as follows. First, EPR method arranges spatial data in the increasing order of proximity by using Hilbert space filling curve, and builds a packed R-tree by bottom-up manner. Second, with packing-based clustering in which arranged spatial data are clustered into continuous disk blocks, EPR method generates spatial data clusters. Third, EPR method distributes EPR index nodes and spatial data clusters on multiple disks through round-robin striping. Experimental results show that EPR method achieves up to 30% or more gains over PR method in query processing speed. In particular, the larger the size of disk blocks is and the smaller the size of spatial data objects is, the better the performance of query processing by EPR method is.

  • PDF

Term Clustering and Duplicate Distribution for Efficient Parallel Information Retrieval (효율적인 병렬정보검색을 위한 색인어 군집화 및 분산저장 기법)

  • 강재호;양재완;정성원;류광렬;권혁철;정상화
    • Journal of KIISE:Software and Applications
    • /
    • v.30 no.1_2
    • /
    • pp.129-139
    • /
    • 2003
  • The PC cluster architecture is considered as a cost-effective alternative to the existing supercomputers for realizing a high-performance information retrieval (IR) system. To implement an efficient IR system on a PC cluster, it is essential to achieve maximum parallelism by having the data appropriately distributed to the local hard disks of the PCs in such a way that the disk I/O and the subsequent computation are distributed as evenly as possible to all the PCs. If the terms in the inverted index file can be classified to closely related clusters, the parallelism can be maximized by distributing them to the PCs in an interleaved manner. One of the goals of this research is the development of methods for automatically clustering the terms based on the likelihood of the terms' co-occurrence in the same query. Also, in this paper, we propose a method for duplicate distribution of inverted index records among the PCs to achieve fault-tolerance as well as dynamic load balancing. Experiments with a large corpus revealed the efficiency and effectiveness of our method.

Design and Implementation of High-Performance Cryptanalysis System Based on GPUDirect RDMA (GPUDirect RDMA 기반의 고성능 암호 분석 시스템 설계 및 구현)

  • Lee, Seokmin;Shin, Youngjoo
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.6
    • /
    • pp.1127-1137
    • /
    • 2022
  • Cryptographic analysis and decryption technology utilizing the parallel operation of GPU has been studied in the direction of shortening the computation time of the password analysis system. These studies focus on optimizing the code to improve the speed of cryptographic analysis operations on a single GPU or simply increasing the number of GPUs to enhance parallel operations. However, using a large number of GPUs without optimization for data transmission causes longer data transmission latency than using a single GPU and increases the overall computation time of the cryptographic analysis system. In this paper, we investigate GPUDirect RDMA and related technologies for high-performance data processing in deep learning or HPC research fields in GPU clustering environments. In addition, we present a method of designing a high-performance cryptanalysis system using the relevant technologies. Furthermore, based on the suggested system topology, we present a method of implementing a cryptanalysis system using password cracking and GPU reduction. Finally, the performance evaluation results are presented according to demonstration of high-performance technology is applied to the implemented cryptanalysis system, and the expected effects of the proposed system design are shown.

An Efficient Clustering Method based on Multi Centroid Set using MapReduce (맵리듀스를 이용한 다중 중심점 집합 기반의 효율적인 클러스터링 방법)

  • Kang, Sungmin;Lee, Seokjoo;Min, Jun-ki
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.7
    • /
    • pp.494-499
    • /
    • 2015
  • As the size of data increases, it becomes important to identify properties by analyzing big data. In this paper, we propose a k-Means based efficient clustering technique, called MCSKMeans (Multi centroid set k-Means), using distributed parallel processing framework MapReduce. A problem with the k-Means algorithm is that the accuracy of clustering depends on initial centroids created randomly. To alleviate this problem, the MCSK-Means algorithm reduces the dependency of initial centroids using sets consisting of k centroids. In addition, we apply the agglomerative hierarchical clustering technique for creating k centroids from centroids in m centroid sets which are the results of the clustering phase. In this paper, we implemented our MCSK-Means based on the MapReduce framework for processing big data efficiently.

Design and Implementation of Algorithms for the Motion Detection of Vehicles using Hierarchical Motion Estimation and Parallel Processing (계층화 모션 추정법과 병렬처리를 이용한 차량 움직임 측정 알고리즘 개발 및 구현)

  • 강경훈;정성태;이상설;남궁문
    • Journal of Korea Multimedia Society
    • /
    • v.6 no.7
    • /
    • pp.1189-1199
    • /
    • 2003
  • This paper presents a new method for the motion detection of vehicles using hierarchical motion estimation and parallel processing. It captures the road image by using a CMOS sensor. It divides the captured image into small blocks and detects the motion of each block by using a block-matching method which is based on a hierarchical motion estimation and parallel processing for the real-time processing. The parallelism is achieved by using tile pipeline and the data flow technique. The proposed method has been implemented by using an embedded system. The proposed block matching algorithm has been implemented on PLDs(Programmable Logic Device) and clustering algorithm has been implemented by ARM processor. Experimental results show that the proposed system detects the motion of vehicles in real-time.

  • PDF

A framework for parallel processing in multiblock flow computations (다중블록 유동해석에서 병렬처리를 위한 시스템의 구조)

  • Park, Sang-Geun;Lee, Geon-U
    • Transactions of the Korean Society of Mechanical Engineers B
    • /
    • v.21 no.8
    • /
    • pp.1024-1033
    • /
    • 1997
  • The past several years have witnessed an ever-increasing acceptance and adoption of parallel processing, both for high performance scientific computing as well as for more general purpose applications. Furthermore with increasing needs to perform the complex flow calculations in an efficient manner, the use of the message passing model on distributed networks has emerged as an important alternative to the expensive supercomputers. This work attempts to provide a generic framework to enable the parallelization of all CFD-related works using the master-slave model. This framework consists of (1) input geometry, (2) domain decomposition, (3) grid generation, (4) flow computations, (5) flow visualization, and (6) output display as the sequential components, but performs computations for (2) to (5) in parallel on the workstation clustering. The flow computations are parallized by having multiple copies of the flow-code to solve a PDE on different spatial regions on different processors, while their flow data are exchanged across the region boundaries, and the solution is time-stepped. The Parallel Virtual Machine (PVM) is used for distributed communication in this work.

Prediction of Sunspot Number Time Series using the Parallel-Structure Fuzzy Systems (병렬구조 퍼지시스템을 이용한 태양흑점 시계열 데이터의 예측)

  • Kim Min-Soo;Chung Chan-Soo
    • The Transactions of the Korean Institute of Electrical Engineers D
    • /
    • v.54 no.6
    • /
    • pp.390-395
    • /
    • 2005
  • Sunspots are dark areas that grow and decay on the lowest level of the sun that is visible from the Earth. Shot-term predictions of solar activity are essential to help plan missions and to design satellites that will survive for their useful lifetimes. This paper presents a parallel-structure fuzzy system(PSFS) for prediction of sunspot number time series. The PSFS consists of a multiple number of component fuzzy systems connected in parallel. Each component fuzzy system in the PSFS predicts future data independently based on its past time series data with different embedding dimension and time delay. An embedding dimension determines the number of inputs of each component fuzzy system and a time delay decides the interval of inputs of the time series. According to the embedding dimension and the time delay, the component fuzzy system takes various input-output pairs. The PSFS determines the final predicted value as an average of all the outputs of the component fuzzy systems in order to reduce error accumulation effect.

An Optimized Approach of Fault Distribution for Debugging in Parallel

  • Srivasatav, Maneesha;Singh, Yogesh;Chauhan, Durg Singh
    • Journal of Information Processing Systems
    • /
    • v.6 no.4
    • /
    • pp.537-552
    • /
    • 2010
  • Software Debugging is the most time consuming and costly process in the software development process. Many techniques have been proposed to isolate different faults in a program thereby creating separate sets of failing program statements. Debugging in parallel is a technique which proposes distribution of a single faulty program segment into many fault focused program slices to be debugged simultaneously by multiple debuggers. In this paper we propose a new technique called Faulty Slice Distribution (FSD) to make parallel debugging more efficient by measuring the time and labor associated with a slice. Using this measure we then distribute these faulty slices evenly among debuggers. For this we propose an algorithm that estimates an optimized group of faulty slices using as a parameter the priority assigned to each slice as computed by value of their complexity. This helps in the efficient merging of two or more slices for distribution among debuggers so that debugging can be performed in parallel. To validate the effectiveness of this proposed technique we explain the process using example.

Implementation of MPI-based WiMAX Base Station for SDR System (SDR 시스템을 위한 MPI 기반 WiMAX 기지국의 구현)

  • Ahn, Chi Young;Kim, Hyo Han;Choi, Seung Won
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.9 no.4
    • /
    • pp.59-67
    • /
    • 2013
  • Compared to the conventional Hardware-oriented base stations, Software Defined Radio (SDR)-based base station provides various advantages especially in flexibility and expandability. It enables the multimode capability required in 4th-generation (4G) environment which aims at a convergence network of various kinds of communication standards. However, since a single base station processes all data required in various multiple waveforms, the SDR base station faces a problem of data processing speed. In this paper, we propose a new concept of SDR base station system which adopts a parallel processing technology of clustering environment. We implemented a WiMAX system with SDR concept which adopts the Message Passing Interface (MPI) technology which enables the speed-up operations. In order to maximize the efficiency of parallel processing in signal processing, we analyze how the algorithm at each of modules is related to data to be processed. Through the implemented system, we show a drastic improvement in operation time due to parallel processing using the proposed MPI technology. In addition, we demonstrate a feasibility of SDR system for 4G or even beyond-4G as well.