• Title/Summary/Keyword: parallel file system

Search Result 72, Processing Time 0.026 seconds

Fips : Dynamic File Prefetching Scheme based on File Access Patterns (Fips : 파일 접근 유형을 고려한 동적 파일 선반입 기법)

  • Lee, Yoon-Young;Kim, Chei-Yol;Seo, Dae-Wha
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.29 no.7
    • /
    • pp.384-393
    • /
    • 2002
  • A Parallel file system is normally used to support excessive file requests from parallel applications in a cluster system, whereas prefetching is useful for improving the file system performance. This paper proposes a new prefetching method, Fips(dynamic File Prefetching Scheme based on file access patterms), that is particularly suitable for parallel scientific applications and multimedia web services in a parallel file system. The proposed prefetching method introduces a dynamic prefetching scheme to predict data blocks precisely in run-time although the file access patterns are irregular. In addition, it includes an algorithm to determine whether and when the prefetching is performed using the current available I/O bandwidth. Experimental results confirmed that the use of the proposed prefetching policy in a parallel file system produced a higher file system performance.

Hybrid Channel Model in Parallel File System (병렬 파일 시스템에서의 하이브리드 채널 모델)

  • Lee, Yoon-Young;Hwangbo, Jun-Hyung;Seo, Dae-Wha
    • The KIPS Transactions:PartA
    • /
    • v.10A no.1
    • /
    • pp.25-34
    • /
    • 2003
  • Parallel file system solves I/O bottleneck to store a file distributedly and read it parallel exchanging messages among computers that is connected multiple computers with high speed networks. However, they do not consider the message characteristics and performances are decreased. Accordingly, the current study proposes the Hybrid Channel model (HCM) as a message-management method, whereby the messages of a parallel file system are classified by a message characteristic between control messages and file data blocks, and the communication channel is divided into a message channel and data channel. The message channel then transfers the control messages through TCP/IP with reliability, while the data channel that is implemented by Virtual Interface Architecture (VIA) transfers the file data blocks at high speed. In tests, the proposed parallel file system that is implemented by HCM exhibited a considerably improved performance.

An Analysis of PVFS Performance Optimization on Small Cluster System (소규모 클러스터 시스템에서의 PVFS 성능 최적화에 관한 연구)

  • Cho, Hyeyoung;Cha, Kwangho;Kim, Sungho
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2007.11a
    • /
    • pp.547-549
    • /
    • 2007
  • Recently with increasing the use of parallel computing and cluster system which was connected high speed network, the interest about distributed and parallel file system is increasing. Specially, there are many researches, which focused on optimizing the performance of distributed and parallel file system for the more efficient use of cluster system. In this paper, we analyzed the performance of PVFS(Parallel Virtual File System) in small cluster system. In addition, to improve the PVFS performance we proposed the chancing the size of flow buffer according to the network speed and we optimized the PVFS performance on small cluster system.

  • PDF

Benchmarks for Performance Testing of MPI-IO on the General Parallel File System (범용 병렬화일 시스템 상에서 MPI-IO 방안의 성능 평가 벤티마크)

  • Park, Seong-Sun
    • The KIPS Transactions:PartA
    • /
    • v.8A no.2
    • /
    • pp.125-132
    • /
    • 2001
  • IBM developed the MPI-IO, we call it MPI-2, on the General Parallel File System. We designed and implemented various Matrix Multiplication Benchmarks to evaluate its performances. The MPI-IO on the General Parallel File System shows four kinds of data access methods : the non-collective and blocking, the collective and blocking, the non-collective and non-blocking, and the split collective operation. In this paper, we propose benchmarks to measure the IO time and the computation time for the data access methods. We describe not only its implementation but also the performance evaluation results.

  • PDF

Design and Implementation of a Communication Module of the Parallel Operating File System based on MISIX (MISIX 기반의 병렬 파일 시스템의 통신 모듈 설계 및 구현)

  • Jin, Sung-Kn;Cho, Jong-Hyun;Kim, Hae-Jin;Seo, Dae-Wha
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.6 no.4
    • /
    • pp.373-382
    • /
    • 2000
  • This paper is concerned with development of a communication module of POFS(Parallel Operating File System), which is the parallel file system to be operated on SPAX computer. SPAX is multiprocessor computer with clustering SMP architecture and being developed by ETRI. The operating system for SPAX is MISIX based on the Chorus microkernel. POFS has client/server architecture basically so that it is important to design a communication module. The communication module is so easily affected by network environment that bad design is the major reason that decreases the portability and performance of the parallel file system. This paper describes the structure and performance of the communication of the POFS. the theme is issued in the course of designing and developing POFS. The communication module of POFS was designed to support the portability and the architecture of parallel file system.

  • PDF

A Disk Allocation Scheme for High-Performance Parallel File System (고성능 병렬화일 시스템을 위한 디스크 할당 방법)

  • Park, Kee-Hyun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.9
    • /
    • pp.2827-2835
    • /
    • 2000
  • In recent years, much attention has been focused on improving I/O devices' processing speed which is essential in such large data processing areas as multimedia data processing. And studies on high-performance parallel file systems are considered to be one of such efforts. In this paper, an efficient disk allocation scheme is proposed for high-performance parallel file systems. In other words, the concept of a parallel disk file's parallelism is defined using data declustering characteristic of a given parallel file. With the concept, an efficient disk allocation scheme is proposed which calculates the appropriate degree of data declustering on disks for each parallel file in order to obtain the maximum throughput when more than one parallel file is used at the same time. Since, calculation for obtaining the maximum throughput is too complex as the number of parallel files increases, an approximate disk allocation algorithm is also proposed in this paper. The approximate algorithm is very simple and especially provides very good results when I/O workload is high. In addition, it has shown that the approximate algorithm provides the optimal disk allocation for the maximum throughput when the arrival rate of I/O requests is infinite.

  • PDF

Design and Implementation of the Parallel Multimedia File System on Fast Ethernet (Fast Ethernet 환경에서 병렬 멀티미디어 파일 시스템의 설계와 구현)

  • Park, Seong-Ho;Kim, Gwang-Mun;Jeong, Gi-Dong
    • The KIPS Transactions:PartB
    • /
    • v.8B no.1
    • /
    • pp.89-97
    • /
    • 2001
  • 대용량 멀티미디어 미디어 서버를 구성함에 있어 I/O 병목현상을 극복하기 위하여 저장 서버들과 제어 서버로 구성되어진 2계층 분산 클러스터 서버구조가 많이 사용된다. 2 계층 분산 클러스터 서버는 부하 균등, 대역폭 관리 및 저장 서버의 관리 측면에서 유리한 반면, 저장 서버와 제어 서버간의 통신 오버헤드를 발생시킨다. 이러한 오버헤드를 줄이기 위해서는 저장 서버에서 읽은 미디어 데이터를 제어 서버를 거치지 않고 직접 클라이언트에 전송할 수 있어야 한다. 그리고, 저장 용량을 확장하거나 손상된 디스크를 교체하는 경우를 대비하여 분산 클러스터 서버는 다양한 성능의 이기종 디스크를 지원하여야 한다. 또한, I/O 장치와 운영체제가 빠르게 발전됨에 따라 미디어 서버는 새로운 I/O 장치 및 운영체제 등에 쉽게 이식될 수 있어야 하고, 응용 소프트웨어 개발자가 시스템의 환경에 따라 블록크기, 데이터 배치정책, 사본 정책 등을 유연하게 조절할 수 있어야 한다. 본 논문에서 위에서 언급한 멀티미디어 서버의 요구를 고려하여 Fast Ethernet 환경에서 병렬 멀티미디어 파일 시스템(PMFS : Parallel Multimedia File System)을 설계 및 구현하고 실험을 통해 PVFS(Parallel Virtual File System)와 성능을 비교 분석하였다. 이 실험의 결과에 따르면 PMFS는 멀티미디어 데이터에 대하여 PVFS보다 3%∼15%의 향상된 성능을 보였다.

  • PDF

Parallel File System for Multimedia Data (Multimedia Data를 위한 병렬 파일 시스템)

  • 박시용;석창규;박성호;김영주;정기동
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2000.04a
    • /
    • pp.600-602
    • /
    • 2000
  • 본 논문에서는 여러 개의 디스크를 클러스트화한 메시지 전송 기반의 병렬 멀티미디어 파일 시스템(PMFS: Parallel Multimedia File System)을 제안하고 설계, 구현하였다. 본 논문에서 구현한 PMFS는 이식성, 유연성 그리고 확장성을 고려한 멀티미디어 데이터를 지원하는 병렬 파일 시스템으로 2계층 분산 클러스트 구조에 적합하다. 그리고 제어 메시지와 TCP를 기반으로 서버들간에 통신을 하고 다양한 방법의 데이터 배치 기법을 제공한다. PMFS의 성능 평가 결과 데이터들이 임의 시작 블록과 DIS배치 기법으로 저장된 경우 가장 좋은 성능을 보였다.

  • PDF

Dual-Cache Scheme in Parallel File System (병렬 파일 시스템에서 이중 캐쉬 구조)

  • Jang, Won-Young;Kim, Chei-Yol;Seo, Dae-Wha
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2001.10a
    • /
    • pp.271-274
    • /
    • 2001
  • 프로세스와 디스크 입출력 속도를 비교해보면, 디스크 입출력의 속도가 휠씬 더 느리다. 따라서 디스크 입출력은 현재의 컴퓨팅 환경에서 병목현상이 되고있다. PFSL(Parallel File System for Linux)은 이런 문제를 해결하기 위한 클러스터링 환경의 병렬 파일 시스템이다. PFSL은 리눅스 머신 상에서 POSIX 스레드 라이브러리를 이용하여 멀티 스레드로 수행된다. 이 논문에서는 PFSL의 성능을 개선하기 위해 클러스터 환경의 작업 부하에 적합하도록 설계한 이중 캐쉬 구조를 소개하고자 한다.

  • PDF

Design and performance evaluation of the software RAID file system in the NOW environment (NOW(Network of Workstations) 환경에서 소프트웨어 RAID 파일 시스템의 설계 및 성능 평가)

  • 김종훈;노삼혁;원유헌
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.22 no.6
    • /
    • pp.1266-1272
    • /
    • 1997
  • Due to the price and performanceof uniprocessor workstations and off-the shelf networking, network of workstations(NOW) ae now a cost-effective parallel processing platform tht is competitive with supercomputers. Meanwhile, current network fiile system protocols rely heavily on a central server to coordinate file activity among client workstations. This central server can become a bottleneck that limits scalibility for environments with large numbers of clients. In this paper, we propsoe a highly reliable and effective software RAID file system on the network of workstation environment. We present results form a trace-driven simulation study that shows that the designed software RAID file system is more effective in the aspect of elapsed time when compared with client/server file systems.

  • PDF