• Title/Summary/Keyword: Distributed Processing

Search Result 2,303, Processing Time 0.03 seconds

Design of Distributed Computer Systems Using Tabu Search Method (Tabu 탐색 기법을 이용한 분산 컴퓨팅 시스템 설계)

  • Hong, Jin-Won;Kim, Jae-Yearn
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.18 no.36
    • /
    • pp.143-152
    • /
    • 1995
  • This paper determines the allocation of computers and data files to minimize the sum of processing and communication costs which occur in processing jobs at each node. The problem of optimally configuring a distributed computer system belongs to the class of NP-Complete problems and the object function of this paper is nonlinear function and is hard to solve. This paper seeks the solution of distributed processing system by Tabu Search. Firstly, it presents the method of generating the starting solution proper to the distributed processing system. Secondly, it develops the method of searching neighborhood solutions. Finally, it determines the Tabu restriction appropriate to the distributed processing system. According to the experimental results, this algorithm solves a sized problems in reasonable time and is effective in the convergence of the solution. The algorithm developed in this paper is also applicable to the general allocation problems of the distributed processing system.

  • PDF

Design of Distributed Processing Framework Based on H-RTGL One-class Classifier for Big Data (빅데이터를 위한 H-RTGL 기반 단일 분류기 분산 처리 프레임워크 설계)

  • Kim, Do Gyun;Choi, Jin Young
    • Journal of Korean Society for Quality Management
    • /
    • v.48 no.4
    • /
    • pp.553-566
    • /
    • 2020
  • Purpose: The purpose of this study was to design a framework for generating one-class classification algorithm based on Hyper-Rectangle(H-RTGL) in a distributed environment connected by network. Methods: At first, we devised one-class classifier based on H-RTGL which can be performed by distributed computing nodes considering model and data parallelism. Then, we also designed facilitating components for execution of distributed processing. In the end, we validate both effectiveness and efficiency of the classifier obtained from the proposed framework by a numerical experiment using data set obtained from UCI machine learning repository. Results: We designed distributed processing framework capable of one-class classification based on H-RTGL in distributed environment consisting of physically separated computing nodes. It includes components for implementation of model and data parallelism, which enables distributed generation of classifier. From a numerical experiment, we could observe that there was no significant change of classification performance assessed by statistical test and elapsed time was reduced due to application of distributed processing in dataset with considerable size. Conclusion: Based on such result, we can conclude that application of distributed processing for generating classifier can preserve classification performance and it can improve the efficiency of classification algorithms. In addition, we suggested an idea for future research directions of this paper as well as limitation of our work.

Optimization and Performance Analysis of Cloud Computing Platform for Distributed Processing of Big Data (대용량 데이터의 분산 처리를 위한 클라우드 컴퓨팅 환경 최적화 및 성능평가)

  • Hong, Seung-Tae;Shin, Young-Sung;Chang, Jae-Woo
    • Spatial Information Research
    • /
    • v.19 no.4
    • /
    • pp.55-71
    • /
    • 2011
  • Recently, interest in cloud computing which provides IT resources as service form in IT field is increasing. As a result, much research has been done on the distributed data processing that store and manage a large amount of data in many servers. Meanwhile, in order to effectively utilize the spatial data which is rapidly increasing day by day with the growth of GIS technology, distributed processing of spatial data using cloud computing is essential. Therefore, in this paper, we review the representative distributed data processing techniques and we analyze the optimization requirements for performance improvement of the distributed processing techniques for a large amount of data. In addition, we uses the Hadoop and we evaluate the performance of the distributed data processing techniques for their optimization requirements.

Development of a CUBRID-Based Distributed Parallel Query Processing System

  • Kim, Hyeong-Il;Yang, HyeonSik;Yoon, Min;Chang, Jae-Woo
    • Journal of Information Processing Systems
    • /
    • v.13 no.3
    • /
    • pp.518-532
    • /
    • 2017
  • Due to the rapid growth of the amount of data, research on bigdata processing has been highlighted. For bigdata processing, CUBRID Shard is able to support query processing in parallel way by dividing the database into a number of CUBRID servers. However, CUBRID Shard can answer a user's query only when the query is required to gain accesses to a single CUBRID server, instead of multiple ones. To solve the problem, in this paper we propose a CUBRID based distributed parallel query processing system that can answer a user's query in parallel and distributed manner. Finally, through the performance evaluation, we show that our proposed system provides 2-3 times better performance on query processing time than the existing CUBRID Shard.

Performance Evaluation of PDP System Using Realtime Network Monitoring (실시간 네트워크 모니터링을 적용한 PDP 시스템의 성능 평가)

  • Song, Eun-Ha;Jeong, Jae-Hong;Jeong, Young-Sik
    • The KIPS Transactions:PartA
    • /
    • v.11A no.3
    • /
    • pp.181-188
    • /
    • 2004
  • PDF(Parallel/Distributed Processing) is an internet-based parallel/distributed processing system that utilizes resources from hosts on the internet in idle state to perform large scale application through parallel processing, thus decreasing the total execution time. In this paper. do propose an adaptive method to be changed network environment at any time using realtime monitoring of host. It is found from experiments that parallel/distributed processing has better performance than its without monitoring as an adaptive strategy, which copy with task delay factor by overload and fault of network, be applicable to the cockpits of task allocation algorithm in PDP.

Matrix-based Filtering and Load-balancing Algorithm for Efficient Similarity Join Query Processing in Distributed Computing Environment (분산 컴퓨팅 환경에서 효율적인 유사 조인 질의 처리를 위한 행렬 기반 필터링 및 부하 분산 알고리즘)

  • Yang, Hyeon-Sik;Jang, Miyoung;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.16 no.7
    • /
    • pp.667-680
    • /
    • 2016
  • As distributed computing platforms like Hadoop MapReduce have been developed, it is necessary to perform the conventional query processing techniques, which have been executed in a single computing machine, in distributed computing environments efficiently. Especially, studies on similarity join query processing in distributed computing environments have been done where similarity join means retrieving all data pairs with high similarity between given two data sets. But the existing similarity join query processing schemes for distributed computing environments have a problem of skewed computing load balance between clusters because they consider only the data transmission cost. In this paper, we propose Matrix-based Load-balancing Algorithm for efficient similarity join query processing in distributed computing environment. In order to uniform load balancing of clusters, the proposed algorithm estimates expected computing cost by using matrix and generates partitions based on the estimated cost. In addition, it can reduce computing loads by filtering out data which are not used in query processing in clusters. Finally, it is shown from our performance evaluation that the proposed algorithm is better on query processing performance than the existing one.

Distributed Image Preprocessing using Object Activation (객체 활성화를 이용한 분산 영상처리)

  • Heo, Jin-Kyoung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.11 no.1
    • /
    • pp.87-92
    • /
    • 2011
  • Server overload is directly proportional to requested image data size in a image processing. If request data are increase then system is overloaded in a image processing system. For the reduce of server bottle neck, we will be able to consider a distributed processing. Simple distributed processing system can solve server bottleneck and system overload but high cost system requirements. In this paper, Proposes a new distributed image processing system. Object activation technology are being grafted on to simple distributed processing system. It can optimize the user of system resources and can reuse idle system resources in network.

Distributed Collision-Resolvable Medium Access Control for Wireless LANs with Interference Cancellation Support

  • Shen, Hu;Lv, Shaohe;Wang, Xiaodong;Zhou, Xingming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.8 no.8
    • /
    • pp.2691-2707
    • /
    • 2014
  • Medium access control is critical in wireless networks for efficient spectrum utilization. In this paper, we introduce a novel collision resolution method based on the technique of known interference cancellation, and propose a new MAC protocol named as CR-MAC, in which AP tries to decode all the collided data packets by combining partial retransmissions and known interference cancellation. As the collided transmissions are fully utilized, less retransmission is required, especially in a crowded network. The NS-2simulation and MATLAB numerical results show that, under various network settings, CR-MAC performs much better than the IEEE 802.11 DCF in terms of the aggregation throughput and the expected packet delay.

Ontology data processing method in distributed semantic web environment (분산 시맨틱웹 환경에서의 온톨로지 데이터 처리 기법 연구)

  • Kim, Byung-Gon;Oh, Sung-Kyun
    • Journal of Digital Contents Society
    • /
    • v.9 no.2
    • /
    • pp.277-284
    • /
    • 2008
  • As the increasing of users' request about internet web service, the importance of ontology to construct semantic web is increasing now. Early Internet data processing was studied in the form of data integration through centralized ontology construction. However, because of distributed environment of internet, when integrating data of distributed site, it is required to integrate data of each site in terms of peer-to-peer data processing for corresponding to fast change of internet. In this paper, in distributed environment, we propose data processing method which construct ontology in each site with ontology language OWL. Furthermore, through relational representation of OWL, we propose the system containing distributed query processing for data constructed in different site with different method.

  • PDF

Dynamic Transaction Processing in Distributed Real-Time Systems (실시간 분산 시스템을 위한 동적 트랜잭션 처리)

  • Yun, Yong-Ik
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.6
    • /
    • pp.738-747
    • /
    • 1999
  • 본 논문에서는 분산 실시간 시스템의 특징인 분산 처리 과정의 신뢰성을 지원하기 위한 동적 트랜잭션 처리 구조를 연구하였다. 실시간 분산 처리 환경에서 동적으로 발생하는 실시간 분산 트랜잭션 처리를 위하여 트랜잭션 내에 필수적인 3가지 언어적 특성들을 제시하였다. 첫째는 트랜잭션 내에 실시간 시스템의 가장 중요한 특징인 시간적인 제약 조건들을 정의 할 수 있는 방안을 제시하고, 둘째는 비동기적인 처리 성격을 지닌 실시간 특성을 고려한 비동기적 트랜잭션 처리 방법을 제시한다. 또한, 분산 처리 과정에서 발생되는 예외 사항들을 처리하기 위하여 긴급성을 고려한 다중레벨 우선순위 스케줄링 (Multi-Level Priotiry Scheduling)이라 부르는 트랜잭션 스케줄링 방안을 제시한다. 그리고, 제시한 실시간 분산 트랜잭션 처리 구조의 타당성 및 가능성을 입증하기 위한 실시간 트랜잭션 처리 과정을 시물레이션을 통하여 제시한 언어적 특성에 대한 고려 사항들을 보여준다.Abstract We propose a dynamic transaction processing model that supports a reliability for distributed real-time processing. For the dynamic processing in distributed real-time transaction systems, we suggest three features that are defined in programming language. First, we propose a specification model to explicitly define the time constraints, needs in real-time distributed processing. Second, we describe an asynchronous transaction processing mechanism based on the real-time characteristics. So, we suggest three communication primitives to support asynchronous transaction processing. Lastly, a scheduling policy based on urgent transaction is suggested to manage the exception occurred during the distributed processing. This scheduling policy is called multi-level priotiry scheduling (MPLS). Based on three features and scheduling policy, we describe a direction to manage a dynamic transaction processing in distributed real-time systems.