• Title/Summary/Keyword: Heterogeneous computing

Search Result 398, Processing Time 0.021 seconds

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Global Internet Computing Environment based on Java (자바를 기반으로 한 글로벌 인터넷 컴퓨팅 환경)

  • Kim, Hui-Cheol;Sin, Pil-Seop;Park, Yeong-Jin;Lee, Yong-Du
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.9
    • /
    • pp.2320-2331
    • /
    • 1999
  • Over the Internet, in order to utilize a collection of idle computers as a parallel computing platform, we propose a new scheme called GICE(Global Internet Computing Environment). GICE is motivated to obtain high programmability, efficient support for heterogeneous computing resources, system scalability, and finally high performance. The programming model of GICE is based on a single address space. GICE is featured with a Java based programming environment, a dynamic resource management scheme, and efficient parallel task scheduling and execution mechanisms. Based on a prototype implementation of GICE, we address the concept, feasibility, complexity and performance of Internet computing.

  • PDF

CORBA-based Mutual Authentication Service for Subjects of Electronic Commerce (전자상거래 주체간 CORBA 기반 상호 인증 서비스)

  • Jang, Gyeong-A;Kim, Tae-Yun
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.26 no.10
    • /
    • pp.1237-1247
    • /
    • 1999
  • 이질적 분산 환경에서 전자 상거래는 신임장을 기반으로 상거래 참여자들에 대한 인증 서비스가 제공되어야 한다. 본 연구에서는 CORBA 보안 명세 1 를 기반으로 전자 상거래 참여자들에 대한 객체 단위 인증 및 권한 부여 기법을 제공하는 상호 인증 서비스 구조를 제안한다. 이 구조는 Kerberos 2 의 인증 기법 및 인증 키 교환 기법으로 전자 상거래 참여자간에 상대 주체의 신원 확인 뿐 아니라 거래 진행 중 취득한 정보의 근원을 파악할 수 있도록 하였다. 또한 Kerberos 기법을 CORBA 플랫폼 기반의 상호 인증 구조로서 분산 환경에 대해 확장하였으므로 키 관리 등 보안 정보 관리에 있어 효율적이다.Abstract Electronic commerce shall provide its subjects with a credential-based authentication service in the heterogeneous distributed computing environment. In this paper, based on CORBA security service specification 1 which OMG defined, we propose the mutual authentication service for subjects of electronic commerce, providing the authentication of object level and the authenticated key exchange. This proposed structure, by Kerberos 2 for the authentication and the authenticated key exchange, assures not only the identification of a partner but also the confidence of origin of business item for negotiations between subjects of electronic commerce. Since our deployed Kerberos is extended to the mutual authentication service based on CORBA platform, it is efficient for security administration to manage the information such as a key management in the heterogeneous distributed computing environment.

A Dynamic toad Redistribution Using Genetic Algorithm in Heterogeneous Systems (유전 알고리즘을 이용한 이질형 시스템에서의 동적 부하재분배)

  • Lee Seong Hoon
    • Journal of Internet Computing and Services
    • /
    • v.5 no.6
    • /
    • pp.93-101
    • /
    • 2004
  • In a receiver-initiated load redistribution algorithm, receiver(underloaded processor) continues to send unnecessary request messages for load transfer until a sender(overloaded processor) is found while the system load is light, Therefore, it yields many problems such as low cpu utilization and system throughput because of inefficient inter-processor communications until the receiver receives an accept message from the sender in this environment. This paper presents an approach based on genetic algorithm(GA) for dynamic load redistribution in heterogeneous distributed systems. In this scheme the processors to which the requests are sent off are determined by the proposed GA to decrease unnecessary request messages. The performance of proposed algorithm shows better than that of the conventional algorithm through various experiments.

  • PDF

Data Framework Design of EDISON 2.0 Digital Platform for Convergence Research

  • Sunggeun Han;Jaegwang Lee;Inho Jeon;Jeongcheol Lee;Hoon Choi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.8
    • /
    • pp.2292-2313
    • /
    • 2023
  • With improving computing performance, various digital platforms are being developed to enable easily utilization of high-performance computing environments. EDISON 1.0 is an online simulation platform widely used in computational science and engineering education. As the research paradigm changes, the demand for developing the EDISON 1.0 platform centered on simulation into the EDISON 2.0 platform centered on data and artificial intelligence is growing. Herein, a data framework, a core module for data-centric research on EDISON 2.0 digital platform, is proposed. The proposed data framework provides the following three functions. First, it provides a data repository suitable for the data lifecycle to increase research reproducibility. Second, it provides a new data model that can integrate, manage, search, and utilize heterogeneous data to support a data-driven interdisciplinary convergence research environment. Finally, it provides an exploratory data analysis (EDA) service and data enrichment using an AI model, both developed to strengthen data reliability and maximize the efficiency and effectiveness of research endeavors. Using the EDISON 2.0 data framework, researchers can conduct interdisciplinary convergence research using heterogeneous data and easily perform data pre-processing through the web-based UI. Further, it presents the opportunity to leverage the derived data obtained through AI technology to gain insights and create new research topics.

3-D Hetero-Integration Technologies for Multifunctional Convergence Systems

  • Lee, Kang-Wook
    • Journal of the Microelectronics and Packaging Society
    • /
    • v.22 no.2
    • /
    • pp.11-19
    • /
    • 2015
  • Since CMOS device scaling has stalled, three-dimensional (3-D) integration allows extending Moore's law to ever high density, higher functionality, higher performance, and more diversed materials and devices to be integrated with lower cost. 3-D integration has many benefits such as increased multi-functionality, increased performance, increased data bandwidth, reduced power, small form factor, reduced packaging volume, because it vertically stacks multiple materials, technologies, and functional components such as processor, memory, sensors, logic, analog, and power ICs into one stacked chip. Anticipated applications start with memory, handheld devices, and high-performance computers and especially extend to multifunctional convengence systems such as cloud networking for internet of things, exascale computing for big data server, electrical vehicle system for future automotive, radioactivity safety system, energy harvesting system and, wireless implantable medical system by flexible heterogeneous integrations involving CMOS, MEMS, sensors and photonic circuits. However, heterogeneous integration of different functional devices has many technical challenges owing to various types of size, thickness, and substrate of different functional devices, because they were fabricated by different technologies. This paper describes new 3-D heterogeneous integration technologies of chip self-assembling stacking and 3-D heterogeneous opto-electronics integration, backside TSV fabrication developed by Tohoku University for multifunctional convergence systems. The paper introduce a high speed sensing, highly parallel processing image sensor system comprising a 3-D stacked image sensor with extremely fast signal sensing and processing speed and a 3-D stacked microprocessor with a self-test and self-repair function for autonomous driving assist fabricated by 3-D heterogeneous integration technologies.

A Study on Secure and Efficient Heterogenous Network Management Scheme based on Static TTP (정적인 TTP 기반의 안전하고 효율적인 이기종 네트워크 관리 기법에 관한 연구)

  • Seo, Dae-Hee;Baek, Jang-Mi;Cho, Dong-Sub
    • Journal of Internet Computing and Services
    • /
    • v.9 no.6
    • /
    • pp.63-72
    • /
    • 2008
  • Recent heterogeneous network management researches on information security, however, deal only with simple management using PKI and could not sufficiently address the different kinds of security problems that could arise in a heterogeneous network. Thus, various security requirements should first be satisfied and a security management protocol should first be developed to achieve a secure heterogeneous network. Hence, in this paper, various secure and effective heterogeneous network management that address security issues, which were merely a consideration in existing studies, are proposed. The proposed scheme for the protection of the user privacy is the central object and static middle objects of the process used to mutual authentication, also if communication between users is required 1-out-2 oblivious transfer to communicate by using secret communication, as well as the effectiveness and security conscious approach. Specially The proposed scheme is designed to enhance security and efficiency related to various services required in heterogeneous network, based on the reliable peripheral devices for TTP. Using Mobile device, which has been applied to electronic commerce transactions in existing schemes, this study also proposed an appropriate management scheme that is suitable for a dynamic environment and setting a temporary group to provide various services.

  • PDF

Space-Sharing Scheduling Schemes for NOW with Heterogeneous Computing Power (이질적 계산 능력을 가진 NOW를 위한 공간 공유 스케쥴링 기법)

  • Kim, Jin-Sung;Shim, Young-Chul
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.27 no.7
    • /
    • pp.650-664
    • /
    • 2000
  • NOW(Network of Workstations) is considered as a platform for running parallel programs by many people. One of the fundamental problems that must be addressed to achieve good performance for parallel programs on NOW is the determination of efficient job scheduling policies. Currently most research on NOW assumes that all the workstations in the NOW have the same processing power. In this paper we consider a NOW in which workstations may have different computing power. We introduce 10 classes of space sharing-based scheduling policies that can be applied to the NOW with heterogeneous computing power. We compare the performance of these scheduling policies by using the simulator which accepts synthetically generated sequential and parallel workloads and generates the response time and waiting time of parallel jobs as performance indices of various scheduling strategies. Through the experiments the case when a parallel program is partitioned heterogeneously in proportion to the computing power of workstations is shown to have better performance than when a parallel program is partitioned into parallel processes of the same size. When the owner returns to the workstation which is executing a parallel process, the policy which just lowers the priority of the parallel process shows better performance than the one which migrates the parallel process to a new idle workstation. Among the policies which use heterogeneous partitioning and process priority lowering, the adaptive policy performed best across the wide range of inter-arrival time of parallel programs but when the load imbalance among parallel processes becomes very high, the modified adaptive policy performed better.

  • PDF

A Novel Processor Allocation Policy for List Scheduling in Distributed Heterogeneous Computing System (분산 이기종 시스템에서 리스트 스케줄링 알고리즘을 위한 새로운 프로세서 할당 정책)

  • Yoon, Wan-Oh;Song, In-Seong;Yoon, Jun-Chol;Choi, Sang-Bang
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.2
    • /
    • pp.76-89
    • /
    • 2010
  • The performance of Distributed Heterogeneous Computing System depends on the algorithm which schedules input DAG graph. Among various scheduling algorithms, list scheduling algorithm provides superior performance with low complexity. List scheduling consists of task prioritizing phase and processor allocation phase, but most studies only focus on task prioritizing phase. In this paper, we propose LIP policy which has the same complexity with traditional allocation policies but has superior performance. The performance of LIP has been observed by applying them to task prioritizing phase of traditional list scheduling algorithms, HCPT, HEFT, GCA, and PETS. The results show that LIP has better performance than insertion-based policy and non-insertion-based policy, which are traditional processor allocation policies.

CPU-GPU2 Trigeneous Computing for Iterative Reconstruction in Computed Tomography

  • Oh, Chanyoung;Yi, Youngmin
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.5 no.4
    • /
    • pp.294-301
    • /
    • 2016
  • In this paper, we present methods to efficiently parallelize iterative 3D image reconstruction by exploiting trigeneous devices (three different types of device) at the same time: a CPU, an integrated GPU, and a discrete GPU. We first present a technique that exploits single instruction multiple data (SIMD) architectures in GPUs. Then, we propose a performance estimation model, based on which we can easily find the optimal data partitioning on trigeneous devices. We found that the performance significantly varies by up to 6.23 times, depending on how SIMD units in GPUs are accessed. Then, by using trigeneous devices and the proposed estimation models, we achieve optimal partitioning and throughput, which corresponds to a 9.4% further improvement, compared to discrete GPU-only execution.