• Title/Summary/Keyword: Cluster Computing

Search Result 427, Processing Time 0.031 seconds

Cost-Effective MapReduce Processing in the Cloud (클라우드 환경에서의 비용 효율적인 맵리듀스 처리)

  • Ryu, Wooseok
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.10a
    • /
    • pp.114-115
    • /
    • 2018
  • This paper studies a mechanism for cost-effective analysis of big data in the cloud environment. Recently, as a storage of electronic medical records can be managed outside the hospital, there is a growing demand for cloud-based big data analysis in small-and-medium hospitals. This paper firstly analyze the Amazon Elastic MapReduce which is a popular cloud framework for big data analysis, and proposes a cost model for analyzing big data using Amazon EMR with less cost. Using the proposed model, the user can construct a cost-effective computing cluster, which maximize the effectiveness of the analysis per operational cost.

  • PDF

Parallel Finite Element Simulation of the Incompressible Navier-stokes Equations (병렬 유한요소 해석기법을 이용한 유동장 해석)

  • Choi H. G.;Kim B. J.;Kang S. W.;Yoo J. Y.
    • 한국전산유체공학회:학술대회논문집
    • /
    • 2002.05a
    • /
    • pp.8-15
    • /
    • 2002
  • For the large scale computation of turbulent flows around an arbitrarily shaped body, a parallel LES (large eddy simulation) code has been recently developed in which domain decomposition method is adopted. METIS and MPI (message Passing interface) libraries are used for domain partitioning and data communication between processors, respectively. For unsteady computation of the incompressible Wavier-Stokes equation, 4-step splitting finite element algorithm [1] is adopted and Smagorinsky or dynamic LES model can be chosen fur the modeling of small eddies in turbulent flows. For the validation and performance-estimation of the parallel code, a three-dimensional laminar flow generated by natural convection inside a cube has been solved. Then, we have solved the turbulent flow around MIRA (Motor Industry Research Association) model at $Re = 2.6\times10^6$, which is based on the model height and inlet free stream velocity, using 32 processors on IBM SMP cluster and compared with the existing experiment.

  • PDF

Detection and Recovery of Failure Node in SAN-based Cluster Shared File System $SANique^{TM}$ (SAN 기반 클러스터 공유 파일 시스템 $SANique^{TM}$의 오류 노드 탐지 및 회복 기법)

  • Lee, Kyu-Woong
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.13 no.12
    • /
    • pp.2609-2617
    • /
    • 2009
  • This paper describes the design overview of shared file system $SANique^{TM}$ and proposes the method for detection of failure node and recovery management algorithm. We also illustrate the characteristics and system architecture of shared file system based on SAN. In order to provide uninterrupted service, the detection and recovery methods are proposed under the all possible system failures and natural disasters. The various kinds of system failures and disasters are characterized and then the detection and recovery method are proposed in each disconnected computing node group.

Implementation and Performance Evaluation of a Software-based DSM Sytem for a Windows-NT Workstations Cluster (Windows-NT 워크스테이션 클러스터를위한 소프트웨어 기반 분산 공유 메모리 시스템의 구현 및 성능 평가)

  • Lee, Jong-U
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.2
    • /
    • pp.176-184
    • /
    • 1999
  • 지금까지의 소프트웨어 기반 분산 공유 메모리(이하 DSM이라 칭함)시스템은 유닉스 워크스테이션 클러스터를 목표로 하는 것이 대부분이었다. 그러나 현재 Windows-NT 는 서버급 시스템과 PC 모두를 위한 운영체제로서 유닉스와 더불어 널리 사용되고 있는 실정이다. 본 논문에서는 Windows-NT 워크스테이션 클러스터 환경을 위한 DSM 시스템을 구현하고, 구현된 DSM 시스템의 성능 평가 결과를 제시한다. 구현된 DSM 시스템은 Win32 API와 표준 실행-시간 라이브러리를 이용해 구현되었기 때문에 모든 Windows-NT 워크스테이션에서 실행 가능하며 , 프로그래머는 몇 라인의 코드 추가만으로 DSM 시스템 상에서 수행되는 병렬 응용 프로그램을 작성할 수 있다. 워크스테이션 간의 상호연결망으로 범용성을 위해 이더넷 LAN을 지원하였고, 아울러 성능 향상을 위해 기가비트 SAN(System Area Network)도 지원하였다. 기가비트 SAN을 위한 하드웨어로는 Dolphin 사의 PCI-SCI 타입 제품인 Clustar를 사용하였다. 우리는 성능 평가를 통해, 구현된 DSM 시스템이 정확히 동작함은 물론 확장성이 뛰어나다는 것을 확인하였다. 특히 , 기가비트 SAN을 사용할 경우 일부 병렬 벤치 마크 프로그램에서는 노드 수 증가에 따라 성능이 거의 선형적으로 향상된다는 것을 알 수 있었다. 본 논문이 기여하는 바는 Windows-NT 기반 소프트웨어 DSM 시스템의 원천 기술을 확보함으로써 향후 Windows-NT 워크스테이션 클러스터 환경에서의 분산 및 병렬 처리 연구에 도움을 줄 수 있다는 점이다.

Performance Evaluation of Low-Powered Computing Cluster Prototype using Mobile Processors (모바일 프로세서 기반 저전력 컴퓨팅 클러스터 프로토타입 성능 분석)

  • Nam, Dukyun;Gu, Gibeom;Park, Chan Yeol;Ryu, Hoon;Kim, Jik-Soo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.230-233
    • /
    • 2015
  • 본 논문에서는 저전력 클러스터 구축을 위해 확장가능성, 저전력 노드 구성, 자율동작 기능 구현, 플러그인을 통한 기능 확장 등 4가지 핵심 추진사항을 도출하고, 모바일 단말기에 사용되는 저전력 프로세서를 이용하여 컴퓨팅 클러스터 프로토타입을 구축했다. 슈퍼컴퓨터 Top500의 성능 측정으로 활용되는 HPL 벤치마크을 이용하여 프로토타입의 성능을 측정 및 분석하고 모바일 프로세서를 이용한 클러스터의 대규모 확장 시 개선되어야 할 사항을 파악했다.

Implementation of a Raspberry-Pi-Sensor Network (라즈베리파이 센서 네트워크 구현)

  • Moon, Sangook
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.10a
    • /
    • pp.915-916
    • /
    • 2014
  • With the upcoming era of internet of things, the study of sensor network has been paid attention. Raspberry pi is a tiny versatile computer system which is able to act as a sensor node in hadoop cluster network. In this paper, we deployed 5 Raspberry pi's to construct an experimental testbed of hadoop sensor network with 5-node map-reduce hadoop software framework. We compared and analyzed the network architecture in terms of efficiency, resource management, and throughput using various parameters. We used a learning machine with support vector machine as test workload. In our experiments, Raspberry pi fulfilled the role of distributed computing sensor node in the sensor network.

  • PDF

Design and Implementation of Incremental Learning Technology for Big Data Mining

  • Min, Byung-Won;Oh, Yong-Sun
    • International Journal of Contents
    • /
    • v.15 no.3
    • /
    • pp.32-38
    • /
    • 2019
  • We usually suffer from difficulties in treating or managing Big Data generated from various digital media and/or sensors using traditional mining techniques. Additionally, there are many problems relative to the lack of memory and the burden of the learning curve, etc. in an increasing capacity of large volumes of text when new data are continuously accumulated because we ineffectively analyze total data including data previously analyzed and collected. In this paper, we propose a general-purpose classifier and its structure to solve these problems. We depart from the current feature-reduction methods and introduce a new scheme that only adopts changed elements when new features are partially accumulated in this free-style learning environment. The incremental learning module built from a gradually progressive formation learns only changed parts of data without any re-processing of current accumulations while traditional methods re-learn total data for every adding or changing of data. Additionally, users can freely merge new data with previous data throughout the resource management procedure whenever re-learning is needed. At the end of this paper, we confirm a good performance of this method in data processing based on the Big Data environment throughout an analysis because of its learning efficiency. Also, comparing this algorithm with those of NB and SVM, we can achieve an accuracy of approximately 95% in all three models. We expect that our method will be a viable substitute for high performance and accuracy relative to large computing systems for Big Data analysis using a PC cluster environment.

Horizon Run 5: the largest cosmological hydrodynamic simulation

  • Kim, Juhan;Shin, Jihye;Snaith, Owain;Lee, Jaehyun;Kim, Yonghwi;Kwon, Oh-Kyung;Park, Chan;Park, Changbom
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.44 no.1
    • /
    • pp.33.2-33.2
    • /
    • 2019
  • Horizon Run 5 is the most massive cosmological hydrodynamic simulation ever performed until now. Owing to the large spatial volume ($717{\times}80{\times}80[cMpc/h]^3$) and the high resolution down to 1 kpc, we may study the cosmological effects on star and galaxy formations over a wide range of mass scales from the dwarf to the cluster. We have modified the public available Ramses code to harness the power of the OpenMP parallelism, which is necessary for running simulations in such a huge KISTI supercomputer called Nurion. We have reached z=2.3 from z=200 for a given simulation period of 50 days using 2500 computing nodes of Nurion. During the simulation run, we have saved snapshot data at 97 redshifts and two light cone space data, which will be used later for the study of various research fields in galaxy formation and cosmology. We will close this talk by listing possible research topics that will play a crucial role in helping us take lead in those areas.

  • PDF

Considering the accuracy and efficiency of the wireless sensor network Support Plan (무선 센서 네트워크에서의 정확도와 효율성을 고려한 기술 지원 방안)

  • You, Sanghyun;Choi, Jaehyun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.10a
    • /
    • pp.96-98
    • /
    • 2014
  • Wireless Sensor Network(WSN) is a wireless real-time information(Acquired from the sensor nodes that have the computing power and wireless communication capabilities.) collected, and to take advantage of processing techniques. Currently it is very diverse, such as environmental monitoring, health care, security, smart home, smart grid applications is that. Thus it is required in the wireless sensor network, the algorithm for the efficient use of the limited energy capacity. Suggested by the algorithm for selecting a cluster head node for a hybrid type and clustered, by comparing the amount of energy remaining and a connection between the nodes In this paper, we aim to increase efficiency and accuracy of the wireless sensor network.

  • PDF

Particle Swarm Optimization in Gated Recurrent Unit Neural Network for Efficient Workload and Resource Management (효율적인 워크로드 및 리소스 관리를 위한 게이트 순환 신경망 입자군집 최적화)

  • Ullah, Farman;Jadhav, Shivani;Yoon, Su-Kyung;Nah, Jeong Eun
    • Journal of the Semiconductor & Display Technology
    • /
    • v.21 no.3
    • /
    • pp.45-49
    • /
    • 2022
  • The fourth industrial revolution, internet of things, and the expansion of online web services have increased an exponential growth and deployment in the number of cloud data centers (CDC). The cloud is emerging as new paradigm for delivering the Internet-based computing services. Due to the dynamic and non-linear workload and availability of the resources is a critical problem for efficient workload and resource management. In this paper, we propose the particle swarm optimization (PSO) based gated recurrent unit (GRU) neural network for efficient prediction the future value of the CPU and memory usage in the cloud data centers. We investigate the hyper-parameters of the GRU for better model to effectively predict the cloud resources. We use the Google Cluster traces to evaluate the aforementioned PSO-GRU prediction. The experimental shows the effectiveness of the proposed algorithm.