• KIPS Transactions on Software and Data Engineering
• /
• v.7 no.5
• /
• pp.177-188
• /
• 2018

The scale of graph data has been increased rapidly because of the growth of mobile Internet applications and the proliferation of social network services. This brings upon the imminent necessity of efficient distributed and parallel graph processing approach since the size of these large-scale graphs are easily over a capacity of a single machine. Currently, there are two popular parallel graph processing approaches, vertex-centric graph processing and block centric processing. While a vertex-centric graph processing approach can easily be applied to the parallel processing system, a block-centric graph processing approach is proposed to compensate the drawbacks of the vertex-centric approach. In these systems, the initial quality of graph partition affects to the overall performance significantly. However, it is a very difficult problem to divide the graph into optimal states at the initial phase. Thus, several dynamic load balancing techniques have been studied that suggest the progressive partitioning during the graph processing time. In this paper, we present a load balancing algorithms for the block-centric graph processing approach where most of dynamic load balancing techniques are focused on vertex-centric systems. Our proposed algorithm focus on an improvement of the graph partition quality by dynamically reassigning blocks in runtime, and suggests block split strategy for escaping local optimum solution.

• KIPS Transactions on Computer and Communication Systems
• /
• v.7 no.1
• /
• pp.9-18
• /
• 2018

Enhancing performance of big data analytics in distributed environment has been issued because most of the big data related applications such as machine learning techniques and streaming services generally utilize distributed computing frameworks. Thus, optimizing performance of those applications at Spark has been actively researched. Since optimizing performance of the applications at distributed environment is challenging because it not only needs optimizing the applications themselves but also requires tuning of the distributed system configuration parameters. Although prior researches made a huge effort to improve execution performance, most of them only focused on one of three performance optimization aspect: application design, system tuning, hardware utilization. Thus, they couldn't handle an orchestration of those aspects. In this paper, we deeply analyze and model the application processing procedure of the Spark. Through the analyzed results, we propose performance optimization schemes for each step of the procedure: inner stage and outer stage. We also propose appropriate partitioning mechanism by analyzing relationship between partitioning parallelism and performance of the applications. We applied those three performance optimization schemes to WordCount, Pagerank, and Kmeans which are basic big data analytics and found nearly 50% performance improvement when all of those schemes are applied.

• Spatial Information Research
• /
• v.11 no.3
• /
• pp.213-226
• /
• 2003

With the increasing demand of processing massive geographic data, conventional GISs based on the single processor architecture appear to be problematic. Especially, performing complex GIS operations on the massive geographic data is very time consuming and even impossible. This is due to the processor speed development does not keep up with the data volume to be processed. In the field of GIS, this PC clustering is one of the emerging technology for handling massive geographic data effectively. In this study, a MPI(Message Passing Interface)-based parallel processing approach was conducted to implement the existing raster GIS operations that typically requires massive geographic data sets in order to improve the processing capabilities and performance. Specially for this research, four types of raster CIS operations that Tomlin(1990) has introduced for systematic analysis of raster GIS operation. A data decomposition method was designed and implemented for selected raster GIS operations.

• The KIPS Transactions:PartA
• /
• v.10A no.6
• /
• pp.635-642
• /
• 2003

The radical development of recent network technology provides the basic foundation which can establish a high speed and cheap cluster system. It is a general trend that conventional cluster systems are built as the system over a fixed level based on stabilized and high speed local networks. A multi-distributed web cluster group is a web cluster model which can obtain high performance, high efficiency and high availability through mutual cooperative works between effective job division and system nodes through parallel performance of a given work and shared memory of SC-Server with low price and low speed system nodes on networks. For this, multi-distributed web cluster group builds a sub-cluster group bound with single imaginary networks of multiple system nodes and uses the web distributed shared memory of system nodes for the effective data transmission within sub-cluster groups. Since the presented model uses a load balancing and parallel computing method of large-scale work required from users, it can maximize the processing efficiency.

• Proceedings of the Korea Contents Association Conference
• /
• 2014.11a
• /
• pp.37-38
• /
• 2014

최근 SNS의 발전으로 인해 데이터의 양이 급격히 증가하였으며, 이에 따라 빅데이터 처리를 위한 분산 DBMS 기반 질의 처리 연구가 활발히 진행되고 있다. 이를 위해 CUBRID는 CUBRID Shard 서비스를 통해 데이터베이스를 shard 단위로 수평 분할하여 각기 다른 물리 노드에 데이터를 분산 저장하도록 지원한다. 그러나 CUBRID Shard는 shard간 데이터가 독립적으로 관리되기 때문에 세타 조인 및 중첩 질의와 같이 다수 서버에서의 테이블 참조가 필요한 질의는 처리가 불가능하다. 따라서 본 논문에서는 분산 CUBRID 상에서 세타 조인 및 중첩 SQL를 지원하는 질의 재구성 기법을 제안한다.

• Annual Conference of KIPS
• /
• 2023.11a
• /
• pp.79-80
• /
• 2023

동기식 분산 딥러닝 기법은 그래디언트 계산 작업을 다수의 워커가 나누어 병렬 처리함으로써 모델 학습 과정을 효율적으로 단축시킨다. 배치 사이즈는 이터레이션 단위로 처리하는 데이터 개수를 의미하며, 학습 속도 및 학습 모델의 품질에 영향을 미치는 중요한 요소이다. 멀티 GPU 환경에서 작동하는 분산 학습의 경우, 가용 GPU 메모리 용량이 커짐에 따라 선택 가능한 배치 사이즈의 상한이 증가한다. 하지만 배치 사이즈가 학습 속도 및 학습 모델 품질에 미치는 영향은 GPU 활용률, 총 에포크 수, 모델 파라미터 개수 등 다양한 변수에 영향을 받으므로 최적값을 찾기 쉽지 않다. 본 연구는 동기식 분산 딥러닝 환경에서 실험을 통해 최적의 배치 사이즈 선택에 영향을 미치는 주요 요인을 분석한다.

• Journal of KIISE:Computer Systems and Theory
• /
• v.26 no.7
• /
• pp.752-765
• /
• 1999

대부분 병렬처리 시스템에서 성능 파라미터는 복잡하고 프로그램의 수행 시 예견할 수 없게 변하기 때문에 컴파일러가 프로그램 수행에 대한 최적의 성능 파라미터들을 컴파일 시에 결정하기가 힘들다. 본 논문은 병렬 처리 시스템의 프로그램 수행 시, 변화하는 시스템 성능 상태에 따라 전체 성능이 최적화로 적응하는 적응 수행 방식을 제안한다. 본 논문에서는 이 적응 수행 방식 중에 적응 프로그램 수행을 위한 이론적인 방법론 및 구현 방법에 대해 제안하고 적응 제어 수행을 위해 프로그램의 데이타 공유 단위에 대한 적응방식(적응 입도 방식)을 사용한다. 적응 프로그램 수행 방식은 프로그램 수행 시 하드웨어와 컴파일러의 도움으로 프로그램 자신이 최적의 성능을 얻을 수 있도록 적응하는 방식이다. 적응 제어 수행을 위해 수행 시에 병렬 분산 공유 메모리 시스템에서 프로세서 간 공유될 수 있은 데이타의 공유 상태에 따라 공유 데이타의 크기를 변화시키는 적응 입도 방식을 적용했다. 적응 입도 방식은 기존의 공유 메모리 시스템의 공유 데이타 단위의 통신 방식에 대단위 데이타의 전송 방식을 사용자의 입장에 투명하게 통합한 방식이다. 시뮬레이션 결과에 의하면 적응 입도 방식에 의해서 하드웨어 분산 공유 메모리 시스템보다 43%까지 성능이 개선되었다. Abstract On parallel machines, in which performance parameters change dynamically in complex and unpredictable ways, it is difficult for compilers to predict the optimal values of the parameters at compile time. Furthermore, these optimal values may change as the program executes. This paper addresses this problem by proposing adaptive execution that makes the program or control execution adapt in response to changes in machine conditions. Adaptive program execution makes it possible for programs to adapt themselves through the collaboration of the hardware and the compiler. For adaptive control execution, we applied the adaptive scheme to the granularity of sharing adaptive granularity. Adaptive granularity is a communication scheme that effectively and transparently integrates bulk transfer into the shared memory paradigm, with a varying granularity depending on the sharing behavior. Simulation results show that adaptive granularity improves performance up to 43% over the hardware implementation of distributed shared memory systems.

• The KIPS Transactions:PartA
• /
• v.8A no.1
• /
• pp.36-41
• /
• 2001

In this paper, we first design an parallel quadratic sieve algorithm for factoring method. We then present parallel factoring algorithm for factoring a large odd integer by repeatedly using the parallel quadratic sieve algorithm based on the divide-and-conquer strategy on SIMD machines with DMM. We show that this algorithm is optimal in view of the product of time and processor numbers.

• The Transactions of the Korea Information Processing Society
• /
• v.3 no.7
• /
• pp.1680-1686
• /
• 1996

The deterministic update method of intensional predicates in a parallel deductive database that deductive database is distributed in a parallel computer architecture in needed. Using updated data from the deterministic update method, a strategy for parallel evaluation of intensional predicates is required. The paper is concerned with an approach to updating parallel deductive database in which very insertion or deletion can be performed in a deterministic way, and an extended parallel semi-naive evaluation algorithm in a parallel computer architecture. After presenting an approach to updating intensional predicates and strategy for parallel evaluation, its implementation is discussed. A parallel deductive database consists of the set of facts being the extensional database and the set of rules being the intensional database. We assume that these sets are distributed in each processor, research how to update intensional predicates and evaluate using the update method. The parallel architecture for the deductive database consists of a set of processors and a message passing network to interconnect these processors.

• Proceedings of the Korean Information Science Society Conference
• /
• 2010.06a
• /
• pp.317-318
• /
• 2010

클라우드 컴퓨팅 서비스를 제공하기 위해서는 클라우드 컴퓨팅에 적합한 데이터 분산 저장 및 병렬 처리가 가능한 IT 인프라 구축이 필수적이다. 이를 위해서 분산 파일 시스템 중 하나인 HDFS(Hadoop File System)와 병렬 데이터 처리를 지원하기 위한 MapReduce 프레임워크 관련 연구가 각광 받고 있다. 하지만 MapReduce 프레임워크를 구성하는 JobTracker 노드는 SPoF(Single Point of Failure)이기 때문에, 작업 도중 JobTracker 노드의 결함이 발생하게 되면 전체 작업이 실패하게 된다. 위와 같은 문제를 해결하기 위해서 본 논문에서는 MapReduce 프레임워크의 JobTracker 노드 결함 발생에 대처할 수 있는 결함허용 메커니즘을 제안하였다.