• Title/Summary/Keyword: Collective I/O

Search Result 12, Processing Time 0.029 seconds

An Analysis of the Performance of Collective I/Os and the Subgroup Method (집합 I/O와 부분군 기법의 성능 분석)

  • Cha, Kwangho;Cho, Hyeyoung;Kim, Sungho
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2007.11a
    • /
    • pp.513-516
    • /
    • 2007
  • Because many scientific applications require large data processing, the importance of parallel I/O has been increasingly recognized. Collective I/O is one of the considerable features of parallel I/O and enables application programmers to easily handle their large data volume. In this paper we measure and analyze the performance of original collective I/Os and the subgroup method, the way of using collective I/O of MPI effectively. From the experimental results, we found that the two kinds of subgroup method showed different performance. In terms of collective write operation, the subgroup method caused the performance degradation. However, the subgroup method for collective read showed good performance with small data size.

  • PDF

Design and Implementation of An I/O System for Irregular Application under Parallel System Environments (병렬 시스템 환경하에서 비정형 응용 프로그램을 위한 입출력 시스템의 설계 및 구현)

  • No, Jae-Chun;Park, Seong-Sun;;Gwon, O-Yeong
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.26 no.11
    • /
    • pp.1318-1332
    • /
    • 1999
  • 본 논문에서는 입출력 응용을 위해 collective I/O 기법을 기반으로 한 실행시간 시스템의 설계, 구현 그리고 그 성능평가를 기술한다. 여기서는 모든 프로세서가 동시에 I/O 요구에 따라 스케쥴링하며 I/O를 수행하는 collective I/O 방안과 프로세서들이 여러 그룹으로 묶이어, 다음 그룹이 데이터를 재배열하는 통신을 수행하는 동안 오직 한 그룹만이 동시에 I/O를 수행하는 pipelined collective I/O 등의 두 가지 설계방안을 살펴본다. Pipelined collective I/O의 전체 과정은 I/O 노드 충돌을 동적으로 줄이기 위해 파이프라인된다. 이상의 설계 부분에서는 동적으로 충돌 관리를 위한 지원을 제공한다. 본 논문에서는 다른 노드의 메모리 영역에 이미 존재하는 데이터를 재 사용하여 I/O 비용을 줄이기 위해 collective I/O 방안에서의 소프트웨어 캐슁 방안과 두 가지 모형에서의 chunking과 온라인 압축방안을 기술한다. 그리고 이상에서 기술한 방안들이 입출력을 위해 높은 성능을 보임을 기술하는데, 이 성능결과는 Intel Paragon과 ASCI/Red teraflops 기계 상에서 실험한 것이다. 그 결과 응용 레벨에서의 bandwidth는 peak point가 55%까지 측정되었다.Abstract In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. We present two designs, namely, "Collective I/O" and "Pipelined Collective I/O". In the first scheme, all processors participate in the I/O simultaneously, making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, so that only one group performs I/O simultaneously, while the next group performs communication to rearrange data, and this entire process is pipelined to reduce I/O node contention dynamically. In other words, the design provides support for dynamic contention management. Then we present a software caching method using collective I/O to reduce I/O cost by reusing data already present in the memory of other nodes. Finally, chunking and on-line compression mechanisms are included in both models. We demonstrate that we can obtain significantly high-performance for I/O above what has been possible so far. The performance results are presented on an Intel Paragon and on the ASCI/Red teraflops machine. Application level I/O bandwidth up to 55% of the peak is observed.he peak is observed.

Collective I/O with Process grouping (프로세스 그룹화를 이용한 집합 I/O)

  • 차광호;홍정우;이지수
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2003.10a
    • /
    • pp.442-444
    • /
    • 2003
  • 병렬 처리를 요구하는 계산 과학 분야의 문제들 중에는 대용량 데이터 처리를 필요로 하는 경우가 많다. 그러나 기존의 파일 시스템을 그대로 병렬처리 환경에 적용하기에는 많은 문제가 따른다. 이를 위해서 병렬처리를 지원하는 파일 시스템에 대한 연구와 개발이 진행되어 오고 있다. 이와 같은 연구 중 하나인 집합 I/O(Collective I/O)를 본 논문에서 다루고자 한다. 이 집합 I/O는 여러 프로세스의 파일 I/O 요청을 효과적으로 처리하는 방법으로 MPl2의 MPI-10에도 포함되어 있다. 본 논문에서는 어플리케이션 프로그램 측면에서 MPI-10의 집합 I/O를 효과적으로 사용하기 위한 방안을 제시하며, 보편적으로 사용되는 NFS를 이용한 클러스터 시스템에서의 실험 결과를 분석한다.

  • PDF

A Parallel I/O System on Workstation Clustering Environment for Irregular Applications (비정형 응용을 위한 워크스테이션 클러스터링 환경에서의 병렬 입출력 시스템)

  • No, Jae-Chun;Park, Sung-Soon;Choudhary, Alok
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.27 no.5
    • /
    • pp.496-505
    • /
    • 2000
  • Clusters of workstations (COW) are becoming an attractive option for parallel scientific computing, a field formerly reserved to the MPPs, because their cost-performance ratio is usuallybetter than that of comparable MPPS, and their hardware and software can be easily enhanced to thelatest generations. In this paper we present the design and implementation of our runtime library forclusters of workstations, called "Collective I/O Clustering". The library provides a friendlyprogramming model for the I/O of irregular applications on clusters of workstations, being completelyintegrated with the underlying communication and I/O system. In the collective I/O clustering, two I/Oconfigurations are possible. In the first I/O configuration, all processors allocated can act as I/Oservers as well as compute nodes. In the second I/O configuration, only a subset of processors canact as I/O servers, The compression and software caching facilities have been incorporated into thecollective 1/0 clustering to optimize the communication and I/O costs. All the performance results wereobtained on the IBM-SP machine, located at Argonne National Labs.

  • PDF

Research for Efficient Massive File I/O on Parallel Programs (병렬 프로그램에서의 효율적인 대용량 파일 입출력 방식의 비교 연구)

  • Hwang, Gyuhyeon;Kim, Youngtae
    • Journal of Internet Computing and Services
    • /
    • v.18 no.2
    • /
    • pp.53-60
    • /
    • 2017
  • Since processors are handling inputs and outputs independently on distributed memory computers, different file input/output methods are used. In this paper, we implemented and compared various file I/O methods to show their efficiency on distributed memory parallel computers. The implemented I/O systems are as following: (i) parallel I/O using NFS, (ii) sequential I/O on the host processor and domain decomposition, (iii) MPI-IO. For performance analysis, we used a separated file server and multiple processors on one or two computational servers. The results show the file I/O with NFS for inputs and sequential output with domain composition for outputs are best efficient respectively. The MPI-IO result shows unexpectedly the lowest performance.

Magnetic field behavior of Bi$_2CaCu_2O_{8+{\delta}}$ Intrinsic Josephson Junctions (Bi$_2Sr_2CaCu_2O_{8+{\delta}}$ Intrinsic 조셉슨 접합의 자기장 효과)

  • Lee, Ju-Yeong;Lee, Hyeon-Ju;Chong, Yeon-Uk;Lee, Su-Yeon;Kim, Jeong-Gu
    • 한국초전도학회:학술대회논문집
    • /
    • v.9
    • /
    • pp.178-184
    • /
    • 1999
  • We have measured I-V characteristics of Bi$_2Sr_2CaCu_2O_{8+{\delta}}$ mesa containing a small number of intrinsic stacked Josephson junctions in a magnetic field. We fabricated mesa with an area of 40${\times}$40 ${\mu}$m$^2$ containing 3${\sim}$20 intrinsic junctions. We applied magnetic field perpendicular to He CuO$_2$ planes up to 5T. We observed flux-flow branches and flux-flow steps in the I-V characteristics which might be due to collective motion of Josephson vortices in the long junction limit. In a parallel field, critical current I$_c$ varies as I$_c$(B) ${\sim}$ exp(-B/B$_0$), where B$_0$ is about 2T, which is consistent with the theoretical model. DC and AC intrinsic Josephson effects are also discussed.

  • PDF

Coherent motion of fluxons in stacked intrinsic Josephson junctions of $Bi_2$$Sr_2$Ca$Cu_2$$O_{8+x}$ single crystals ($Bi_2Sr_2CaCu_2O_{8+x}$ 단결정 선천성 조셉슨 접합에서의 플럭손 결맞음 운동)

  • Doh, Yong-Joo;Chang, Hyun-Sik;Chang, Dong-In;Lee, Hu-Jong;Kim, Jinhee;Kim, Kyu-Tae;Lee, Woo;Choy, Jin-Ho
    • Progress in Superconductivity
    • /
    • v.3 no.1
    • /
    • pp.28-30
    • /
    • 2001
  • We studied the flux-flow current-voltage characteristics of microwave-generated fluxons formed in serially stacked intrinsic Josephson junctions fabricated on$ HgI_2$-intercalated $Bi_2$$Sr_2$$CaCu_2$O/8+x/(Bi2212) single crystals. With increasing the irradiation power of 73$\square$76 GHz microwave, the supercurrent branch became resistive and split into multiple sub -branches. Each sub-branch represented a specific mode of collective motion of Josephson fluxons. We also observed similar branch splitting In a mesa prepared on an underdoped Bi2212 single crystal in a static magnetic field.

  • PDF

Performance evaluation of collective I/O on an SMP supercomputer (SMP 슈퍼컴퓨터에서의 집합 IO 성능)

  • Cha, Kwangho;Kim, Sungho;Lee, Sik
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2010.11a
    • /
    • pp.1732-1734
    • /
    • 2010
  • 멀티 코어 또는 매니 코어 기반의 HPC 시스템 보급이 늘어나면서 HPC 어플리케이션이 사용하는 프로세스의 수 또한 증가하고 있다. 이런 경우, 기존의 IO 방식이 아닌 병렬 IO 의 사용을 고려하여야 하는데 그 중 특히 집합 IO 는 중요한 역할을 수행한다. 본 연구에서는 IBM Power 595 기반 슈퍼 컴퓨터에서 집합 IO 특성을 알아 본다.

Coherent motion of microwave-induced fluxons in intrinsic Josephson junctions of HgI$_2$-intercalated Bi$_2$Sr$_2$C aCu$_2$O$_{8+x}$ single crystals

  • Kim, Jin-Hee;Doh, Yong-Joo;Chang, Sung-Ho;Lee, Hu-Jong;Chang, Hyun-Sik;Kim, Kyu-Tae;Jang, Eue-Soon;Choy, Jin-Ho
    • 한국초전도학회:학술대회논문집
    • /
    • v.10
    • /
    • pp.65-65
    • /
    • 2000
  • Microwave response of intrinsic Josephson junctions in mesa structure formed on HgI2-intercalated Bi2Sr2CaCu2O8+x single crystals was studied in a wide range of microwave frequency. With irradiation of 73${\sim}$76 GHz microwave, the supercurrent branch becomes resistive above a certain onset microwave power. At low current bias, the current-voltage characteristics show linear behavior, while at high current bias, the resistive branch splits into multiple sub-branches. The voltage spacing between neighboring sub-branches increase with the microwave power and the total number of sub-branches is almost identical to the number of intrinsic Josephson junctions in the mesa. All the experimental results suggest that each sub-branch represents a specific mode of collective motion of Josephson vortices generated by the microwave irradiation. With irradiation of microwave of microwave of frequency lower than 20 GHz, on the other hand, no branch splitting was observed and the current-voltage characteristics exhibited complex behavior at hlgh blas currents. This result can be explained in terms of incoherent motion of Josephson vortices generated by non-uniform microwave irradiation.

  • PDF

Video Retrieval System supporting Adaptive Streaming Service (적응형 스트리밍 서비스를 지원하는 비디오 검색 시스템)

  • 이윤채;전형수;장옥배
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.9 no.1
    • /
    • pp.1-12
    • /
    • 2003
  • Recently, many researches into distributed processing on Internet, and multimedia data processing have been performed. Rapid and convenient multimedia services supplied with high quality and high speed are to be needed. In this paper, we design and implement clip-based video retrieval system on the Web enviroment in real-time. Our system consists of the content-based indexing system supporting convenient services for video content providers, and the Web-based retrieval system in order to make it easy and various information retrieval for users in the Web. Three important methods are used in the content-based indexing system, key frame extracting method by dividing video data, clip file creation method by clustering related information, and video database construction method by using clip unit. In Web-based retrieval system, retrieval method ny using a key word, two dimension browsing method of key frame, and real-time display method of the clip are used. In this paper, we design and implement the system that supports real-time display method of the clip are used. In this paper, we design and implement the system that supports real-time retrieval for video clips on Web environment and provides the multimedia service in stability. The proposed methods show a usefulness of video content providing, and provide an easy method for serching intented video content.