• Title/Summary/Keyword: Cluster Computing

Search Result 427, Processing Time 0.023 seconds

Design of MAHA Supercomputing System for Human Genome Analysis (대용량 유전체 분석을 위한 고성능 컴퓨팅 시스템 MAHA)

  • Kim, Young Woo;Kim, Hong-Yeon;Bae, Seungjo;Kim, Hag-Young;Woo, Young-Choon;Park, Soo-Jun;Choi, Wan
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.81-90
    • /
    • 2013
  • During the past decade, many changes and attempts have been tried and are continued developing new technologies in the computing area. The brick wall in computing area, especially power wall, changes computing paradigm from computing hardwares including processor and system architecture to programming environment and application usage. The high performance computing (HPC) area, especially, has been experienced catastrophic changes, and it is now considered as a key to the national competitiveness. In the late 2000's, many leading countries rushed to develop Exascale supercomputing systems, and as a results tens of PetaFLOPS system are prevalent now. In Korea, ICT is well developed and Korea is considered as a one of leading countries in the world, but not for supercomputing area. In this paper, we describe architecture design of MAHA supercomputing system which is aimed to develop 300 TeraFLOPS system for bio-informatics applications like human genome analysis and protein-protein docking. MAHA supercomputing system is consists of four major parts - computing hardware, file system, system software and bio-applications. MAHA supercomputing system is designed to utilize heterogeneous computing accelerators (co-processors like GPGPUs and MICs) to get more performance/$, performance/area, and performance/power. To provide high speed data movement and large capacity, MAHA file system is designed to have asymmetric cluster architecture, and consists of metadata server, data server, and client file system on top of SSD and MAID storage servers. MAHA system softwares are designed to provide user-friendliness and easy-to-use based on integrated system management component - like Bio Workflow management, Integrated Cluster management and Heterogeneous Resource management. MAHA supercomputing system was first installed in Dec., 2011. The theoretical performance of MAHA system was 50 TeraFLOPS and measured performance of 30.3 TeraFLOPS with 32 computing nodes. MAHA system will be upgraded to have 100 TeraFLOPS performance at Jan., 2013.

A Content-Aware toad Balancing Technique Based on Histogram Transformation in a Cluster Web Server (클러스터 웹 서버 상에서 히스토그램 변환을 이용한 내용 기반 부하 분산 기법)

  • Hong Gi Ho;Kwon Chun Ja;Choi Hwang Kyu
    • Journal of Internet Computing and Services
    • /
    • v.6 no.2
    • /
    • pp.69-84
    • /
    • 2005
  • As the Internet users are increasing rapidly, a cluster web server system is attracted by many researchers and Internet service providers. The cluster web server has been developed to efficiently support a larger number of users as well as to provide high scalable and available system. In order to provide the high performance in the cluster web server, efficient load distribution is important, and recently many content-aware request distribution techniques have been proposed. In this paper, we propose a new content-aware load balancing technique that can evenly distribute the workload to each node in the cluster web server. The proposed technique is based on the hash histogram transformation, in which each URL entry of the web log file is hashed, and the access frequency and file size are accumulated as a histogram. Each user request is assigned into a node by mapping of (hashed value-server node) in the histogram transformation. In the proposed technique, the histogram is updated periodically and then the even distribution of user requests can be maintained continuously. In addition to the load balancing, our technique can exploit the cache effect to improve the performance. The simulation results show that the performance of our technique is quite better than that of the traditional round-robin method and we can improve the performance more than $10\%$ compared with the existing workload-aware load balancing(WARD) method.

  • PDF

The Technology Trend of Interconnection Network for High Performance Computing (고성능 컴퓨팅을 위한 인터커넥션 네트워크 기술 동향)

  • Cho, Hyeyoung;Jun, Tae Joon;Han, Jiyong
    • Journal of the Korea Convergence Society
    • /
    • v.8 no.8
    • /
    • pp.9-15
    • /
    • 2017
  • With the development of semiconductor integration technology, central processing units and storage devices have been miniaturized and performance has been rapidly developed, interconnection network technology is becoming a more important factor in terms of the performance of high performance computing system. In this paper, we analyze the trend of interconnection network technology used in high performance computing. Interconnect technology, which is the most widely used in the Supercomputer Top 500(2017. 06.), is an Infiniband. Recently, Ethernet is the second highest share after InfiniBand due to the emergence of 40/100Gbps Gigabit Ethernet technology. Gigabit Ethernet, where latency performance is lower than InfiniBand, is preferred in cost-effective medium-sized data centers. In addition, top-end HPC systems that demand high performance are devoting themselves from Ethernet and InfiniBand technologies and are attempting to maximize system performance by introducing their own interconnect networks. In the future, high-performance interconnects are expected to utilize silicon-based optical communication technology to exchange data with light.

Design and Implementation of HPC Job Management Framework for Computational Scientific Simulation (계산과학 시뮬레이션을 위한 HPC 작업 관리 프레임워크의 설계 및 구현)

  • Yu, Jung-Lok;Kim, Han-Gi;Byun, Hee-Jung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.05a
    • /
    • pp.554-557
    • /
    • 2016
  • Recently, supercomputer has been increasingly adopted as a computing environment for scientific simulation as well as education, healthcare and national defence. Especially, supercomputing system with heterogeneous computing resources is gaining resurgence of interest as a next-generation problem solving environment, allowing theoretical and/or experimental research in various fields to be free of time and spatial limits. However, traditional supercomputing services have only been handled through a simple form of command-line based console, which leads to the critical limit of accessibility and usability of heterogeneous computing resources. To address this problem, in this paper, we provide the design and implementation of web-based HPC (High Performance Computing) job management framework for computational scientific simulation. The proposed framework has highly extensible design principles, providing the abstraction interfaces of job scheduler (as well as bundle scheduler plug-ins for LoadLeveler, Sun Grid Engine, OpenPBS scheduler) in order to easily incorporate the broad spectrum of heterogeneous computing resources such as cluster, computing cloud and grid. We also present the detailed specification of HTTP standard based RESTful endpoints, which manage simulation job's life-cycles such as job creation, submission, control and status monitoring, etc., enabling various 3rd-party applications to be newly created on top of the proposed framework.

  • PDF

A Hierarchical Clustering Technique of XML Documents based on Representative Path (대표 경로에 기반한 XML 문서의 계층 군집화 기법)

  • Kim, Woo-Saeng
    • Journal of Internet Computing and Services
    • /
    • v.10 no.3
    • /
    • pp.141-150
    • /
    • 2009
  • XML is increasingly important in data exchange and information management. A large amount of efforts have been spent in developing efficient techniques for accessing, querying, and storing XML documents. In this paper, we propose a new method to cluster XML documents efficiently. A new prepresentative path called a virtul path which can represent both the structure and the contents of a XML document is proposed for the feature of a XML document. A method to apply the well known hierarchical clustering techniques to the representative paths to cluster XML documents is also proposed. The experiment shows that the true clusters are formed in a compact shape when a virtual path is used for the feature of a XML document.

  • PDF

Automatic Dynamic Range Improvement Method using Histogram Modification and K-means Clustering (히스토그램 변형 및 K-means 분류 기반 동적 범위 개선 기법)

  • Cha, Su-Ram;Kim, Jeong-Tae;Kim, Min-Seok
    • Journal of Broadcast Engineering
    • /
    • v.16 no.6
    • /
    • pp.1047-1057
    • /
    • 2011
  • In this paper, we propose a novel tone mapping method that implements histogram modification framework on two local regions that are classified using K-means clustering algorithm. In addition, we propose automatic parameter tuning method for histogram modification. The proposed method enhances local details better than the global histogram method. Moreover, the proposed method is fully automatic in the sense that it does not require intervention from human to tune parameters that are involved for computing tone mapping functions. In simulations and experimental studies, the proposed method showed better performance than existing histogram modification method.

Basic Prototype Design and Verification of Hadoop Cluster based on Private Cloud Infrastructure for SMB (중소기업을 위한 프라이빗 클라우드 인프라 기반 하둡 클러스터의 기본 프로토타입 설계 및 실증)

  • Cha, Byung-Rae;Kim, Hyeong-Gyun;Kim, Dae-Gue;Kim, Jong-Won;Kim, Yong-Il
    • Journal of Advanced Navigation Technology
    • /
    • v.17 no.2
    • /
    • pp.225-233
    • /
    • 2013
  • Recently, Cloud Computing and Big Data has become a buzzword in the field of IT. In this paper, as part of special efforts to support small businesses (SMB) in these situations, we designed the basic prototypes ver. 0.1, 0.2, and 0.5 for Hadoop cluster based on private cloud infrastructure and implemented the part of basic prototypes. And we verified the performances of the basic prototypes using ASA Dataset.

Application Layer Multicast Tree Constructing Algorithm for Real-time Media Delivery (실시간 미디어 전송을 위한 응용계층 멀티캐스트 트리 구성 알고리즘)

  • Song Hwangjun;Lee Dong Sup
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.29 no.11B
    • /
    • pp.991-1000
    • /
    • 2004
  • This paper presents an application layer multicast tree constructing algorithm to minimize the average time delay from the sender to end-systems for the effective real-time media delivery. Simultaneously, the proposed algorithm takes into account the computing power and the network condition of each end-system as a control variable and thus avoids the undesirable case that loads are concentrated to only several end-systems. The multicast tree is constructed by clustering technique and modified Dijkstra's algorithm in two steps, i.e. tree among proxy-senders and tree in each cluster. By the experimental results, we show that the proposed algorithm can provide an effective solution.

Autonomic Self Healing-Based Load Assessment for Load Division in OKKAM Backbone Cluster

  • Chaudhry, Junaid Ahsenali
    • Journal of Information Processing Systems
    • /
    • v.5 no.2
    • /
    • pp.69-76
    • /
    • 2009
  • Self healing systems are considered as cognation-enabled sub form of fault tolerance system. But our experiments that we report in this paper show that self healing systems can be used for performance optimization, configuration management, access control management and bunch of other functions. The exponential complexity that results from interaction between autonomic systems and users (software and human users) has hindered the deployment and user of intelligent systems for a while now. We show that if that exceptional complexity is converted into self-growing knowledge (policies in our case), can make up for initial development cost of building an intelligent system. In this paper, we report the application of AHSEN (Autonomic Healing-based Self management Engine) to in OKKAM Project infrastructure backbone cluster that mimics the web service based architecture of u-Zone gateway infrastructure. The 'blind' load division on per-request bases is not optimal for distributed and performance hungry infrastructure such as OKKAM. The approach adopted assesses the active threads on the virtual machine and does resource estimates for active processes. The availability of a certain server is represented through worker modules at load server. Our simulation results on the OKKAM infrastructure show that the self healing significantly improves the performance and clearly demarcates the logical ambiguities in contemporary designs of self healing infrastructures proposed for large scale computing infrastructures.

An Implementation of Animated GIF Generating and Viewing Application Using Mobile-JPVM (Mobile-JPVM을 이용한 Animated GIF 생성 및 뷰잉 프로그램 구현)

  • Lee, Ye-In;Lee, Jong-Woo
    • Journal of Digital Contents Society
    • /
    • v.10 no.4
    • /
    • pp.485-492
    • /
    • 2009
  • In these days mobile handsets have come to be used at almost every user. The mobility of mobile devices and the performance improvement of the mobile networks have made this trend possible. As a great variety of mobile applications are published, the necessity of running large-scale mobile applications becomes greater than before. To accomplish this, the existing researchers have developed mobile cluster computing libraries like Mobile-JPVM. In this paper, we implement a compute-intensive Animated GIF generating application and its cell phone viewer software using Mobile-JPVM library. We find out by the real execution of our softwares on the KTF handsets that they can sufficiently run on cellular phones. Our Animated GIF generator and its viewer are going to be commercially used for the mobile fashion advertisement systems.

  • PDF