• Title/Summary/Keyword: Sun Grid Engine

Search Result 9, Processing Time 0.025 seconds

Optimization and Stabilization of Satellite Data Distributed Processing System (위성 데이터 분산처리 시스템 최적화 및 안정화)

  • Choi, Yun-Soo;Lee, Won-Goo;Lee, Min-Ho;Kim, Sun-Tae;Lee, Sang-Hwan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.18 no.11
    • /
    • pp.13-21
    • /
    • 2013
  • The goal of this paper is to provide performance improvement and stability for satellite data correction of some distortions due to cloud or radiance through distributed processing on cluster. To do this, we proposed and implemented SGE(Sun Grid Engine) based distributed processing methods using local storages and a status table. In the verification, the experiment result revealed that the proposed system on seven nodes improved the processing speed by 138.81% as compare to the existing system and provided good stability as well. This result showed that the proposed distributed processing work is more appropriate to process CPU bound jobs than I/O bound jobs. We expect that the proposed system will give scientists improved analysis performance in various fields and near-real time analysis services.

Sun Grid Engine-based Satellite Image Processing System (Sun Grid Engine 기반 위성영상 처리 시스템)

  • Choi, Yun-soo;Lee, Min-ho;Lee, Sang-hwan
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.05a
    • /
    • pp.61-63
    • /
    • 2013
  • 환경, 해양, 지질자원, 기상 기후 등 다양한 분야에서 연구목적으로 인공위성에 의해 관측된 위성영상을 활용하고 있다. 인공위성으로부터 수신된 영상데이터는 많은 왜곡을 포함하고 있으므로, 이를 이용하여 신뢰할 수 있는 분석을 하기 위해서는 영상데이터에 대한 보정 작업이 선행되어야 한다. 하지만 대부분의 전처리 작업이 단일노드/단일코어를 이용하고 있기 때문에 많은 연구자들이 불편함을 가지고 있다. 본 논문은 단일노드/단일코어 상에서 수행되는 위성영상 처리 방식을 다중노드/다중코어를 이용하는 분산 처리 방안을 제안한다. 7개의 작업 실행 노드를 이용하여 실험한 결과에서, 분산 기반의 처리 성능이 단일노드/단일코어에서 수행된 처리 성능보다 최고 560.65% 향상되었음을 보여주었다.

Improving the Job Success Rate through Analysis of User Logs in HPC (HPC 환경에서 사용자 로그 분석을 통한 작업 성공률 개선)

  • Yoon, JunWeon;Hong, TaeYoung;Kong, Ki-Sik;Park, ChanYeol
    • Journal of Digital Contents Society
    • /
    • v.16 no.5
    • /
    • pp.691-697
    • /
    • 2015
  • Supercomputers are used for many different areas including new product design of industries as well as state-of-the-art science and technology for large amount of computational needs. Tachyon is a 4th supercomputer built at KISTI that is a high-performance parallel computing system with 3,200 computing nodes and infrastructures. This system is currently about 10,000 users and over 170 organizations are used, the number of jobs they are performing work in batch type form through a scheduler. Also, this system logs lots of job scripts, execution environment, library, job status from the job submit to end. In this paper, we analyzed batch jobs information from Sun Grid Engine, that use as a scheduler in Tachyon system, and job executed information in Tachyon System. In particular, we distinguished the fail jobs from the all tasks that users perform and we analyzed the cause of failure. Among them, we can extracted some of jobs that can be regarded as normal jobs through the improvement in those works logged as all of fail jobs.

A Study on the Accuracy of CFD Prediction for Small Scaled 4 Nozzle Clustered Engine Using Air (공기를 이용한 축소형 4노즐 클러스터드 엔진 저부 유동의 CFD 해석 검증)

  • Kim, Seong-Lyong;Kim, In-Sun
    • Proceedings of the Korean Society of Propulsion Engineers Conference
    • /
    • 2011.11a
    • /
    • pp.78-84
    • /
    • 2011
  • CFD simulation has been conducted on a small scaled 4 nozzle clustered engine operating with air. In the present paper, the effects of grid size, turbulence models, flux difference methods have been compared. The results show that the base flows are somewhat different as the turbulence models, while Roe and AUSM flux differences produced almost the same results. Spalart-Allmaras turbulence model produces more accurate results rather than famous SST k-w model. The calculated Mach number and pressure profile in the engine base reveal the complex base flow structure, which is somewhat different from the generally estimated flow fields.

  • PDF

Evaluating the Scalability of Distributed Satellite Data Processing System (위성 데이터 분산 처리 시스템의 확장성 평가)

  • Choi, Yun-Soo;Lee, Min-Ho;Lee, Sang-Hwan
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2013.07a
    • /
    • pp.395-397
    • /
    • 2013
  • MODIS는 기상, 대기, 해양, 그리고 육상 등의 지구전체에 대한 정보를 산출하기 위한 센서로서, 인공위성에 탑재되어 지구관측 데이터를 생산한다. 최초의 MODIS 위성 데이터는 많은 왜곡을 포함하고 있으므로 지형 및 광휘 보정작업은 분석 작업을 하기 위한 필수적인 전처리 작업이다. 위성 데이터 처리를 위해 개발된 SeaDAS는 단일노드/단일코어상에서 수행되기 적합하게 개발되었기 때문에, 대용량의 위성데이터를 전처리하기 위해 많은 시간을 소비해야 한다. 본 논문은 Sun Grid Engine 기반의 다중노드/다중코어를 이용하는 위성 데이터 분산 처리 방법을 제안하고 성능 및 확장성에 대한 평가를 수행한다.

  • PDF

Design and Implementation of Ganga Plugins for Remote SGE (원격 SGE(Sun Grid Engine) 연동을 위한 Ganga 플러그인 설계 및 구현)

  • Kim, Han-Gi;Hwang, Soon-Wook
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2010.06b
    • /
    • pp.523-526
    • /
    • 2010
  • 고에너지 물리나 바이오 인포매틱스 연구 분야는 대량의 컴퓨팅 자원을 필요로 한다. 이를 위해서 클러스터 컴퓨팅, 그리드 컴퓨팅, 슈퍼 컴퓨팅, 클라우드 컴퓨팅 자원 등의 여러 컴퓨팅 자원이 사용된다. 사용자는 각 컴퓨팅 자원을 이용하기 위해서 각각의 사용 방법을 배우고, 저마다의 방법으로 각 컴퓨팅 자원을 사용한다. 사용자가 각 컴퓨터 자원의 사용 방법을 습득하기 위해서는 많은 비용과 시간이 필요하다. 사용자가 보다 쉽고 빠르게 컴퓨팅 자원을 활용하기 위하여 여러 컴퓨팅 자원을 사용하기 위한 동일 인터페이스 개발이 필요하다. 본 논문에서는 한국과학기술정보연구원의 타키온 슈퍼컴퓨터를 이용하는 사용자에게 동일한 사용자 인터페이스를 제공하기 위한 Ganga의 원격 SGE 백엔드(Backend) 플러그인을 설계하고 구현하였다.

  • PDF

A Study of Applicability of a RNG $k-\varepsilon$ Model (RNG $k-\varepsilon$ 모델의 적용성에 대한 연구)

  • Yang, Hei-Cheon;Ryou, Hong-Sun;Lim, Jong-Han
    • Transactions of the Korean Society of Mechanical Engineers B
    • /
    • v.21 no.9
    • /
    • pp.1149-1164
    • /
    • 1997
  • In this study, the applicability of the RNG k-.epsilon. model to the analysis of the complex flows is studied. The governing equations based on a non-orthogonal coordinate formulation with Cartesian velocity components are used and discretized by the finite volume method with non-staggered variable arrangements. The predicted results using the RNG k-.epsilon. model of three complex flows, i.e., the flow over a backward-facing step and a blunt flat plate, the flow around a 2D model car are compared to these from the standard k-.epsilon. model and experimental data. That of the unsteady axisymmetric turbulent flow within a cylinder of reciprocating model engine including port/valve assembly and the spray characteristics within a chamber of direct injection model engine are compared to these from the standard k-.epsilon. model and experimental data. The results of reattachment length, separated eddy size, average surface pressure distribution using the RNG k-.epsilon. model show more reasonable trends comparing with the experimental data than those using the modified k-.epsilon. model. Although the predicted rms velocity using the modified k-.epsilon. model is lower considerably than the experimental data in incylinder flow with poppet valve, predicted axial and radial velocity distributions at the valve exit and in-cylinder region show good agreements with the experimental data. The spray tip penetration predicted using the RNG k-.epsilon. model is more close to the experimental data than that using the modified k-.epsilon. model. The application of the RNG k-.epsilon. model seems to have some potential for the simulations of the unsteady turbulent flow within a port/valve-cylinder assembly and the spray characteristics over the modified k-.epsilon. model.

DEVELOPMENT OF REAL-TIME DATA REDUCTION PIPELINE FOR KMTNet (KMTNet 실시간 자료처리 파이프라인 개발)

  • Kim, D.J.;Lee, C.U.;Kim, S.L.;Park, B.G.
    • Publications of The Korean Astronomical Society
    • /
    • v.28 no.1
    • /
    • pp.1-6
    • /
    • 2013
  • Real-time data reduction pipeline for the Korea Microlensing Telescope Network (KMTNet) was developed by Korea Astronomy and Space Science Institute (KASI). The main goal of the data reduction pipeline is to find variable objects and to record their light variation from the large amount of observation data of about 200 GB per night per site. To achieve the goal we adopt three strategic implementations: precision pointing of telescope using the cross correlation correction for target fields, realtime data transferring using kernel-level file handling and high speed network, and segment data processing architecture using the Sun-Grid engine. We tested performance of the pipeline using simulated data which represent the similar circumstance to CTIO (Cerro Tololo Inter-American Observatory), and we have found that it takes about eight hours for whole processing of one-night data. Therefore we conclude that the pipeline works without problem in real-time if the network speed is high enough, e.g., as high as in CTIO.

Design and Implementation of HPC Job Management Framework for Computational Scientific Simulation (계산과학 시뮬레이션을 위한 HPC 작업 관리 프레임워크의 설계 및 구현)

  • Yu, Jung-Lok;Kim, Han-Gi;Byun, Hee-Jung
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.05a
    • /
    • pp.554-557
    • /
    • 2016
  • Recently, supercomputer has been increasingly adopted as a computing environment for scientific simulation as well as education, healthcare and national defence. Especially, supercomputing system with heterogeneous computing resources is gaining resurgence of interest as a next-generation problem solving environment, allowing theoretical and/or experimental research in various fields to be free of time and spatial limits. However, traditional supercomputing services have only been handled through a simple form of command-line based console, which leads to the critical limit of accessibility and usability of heterogeneous computing resources. To address this problem, in this paper, we provide the design and implementation of web-based HPC (High Performance Computing) job management framework for computational scientific simulation. The proposed framework has highly extensible design principles, providing the abstraction interfaces of job scheduler (as well as bundle scheduler plug-ins for LoadLeveler, Sun Grid Engine, OpenPBS scheduler) in order to easily incorporate the broad spectrum of heterogeneous computing resources such as cluster, computing cloud and grid. We also present the detailed specification of HTTP standard based RESTful endpoints, which manage simulation job's life-cycles such as job creation, submission, control and status monitoring, etc., enabling various 3rd-party applications to be newly created on top of the proposed framework.

  • PDF